-- Logs begin at Tue 2022-08-09 18:14:20 UTC, end at Tue 2022-08-09 19:31:27 UTC. -- Aug 09 18:59:35.385289 dv-cinder-852654-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 09 18:59:35.396429 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 9 18:59:35 2022] *** Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: nodename: dv-cinder-852654-1-iscsi Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: machine: x86_64 Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: clock source: unix Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: pcre jit disabled Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: detected number of CPU cores: 4 Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: current working directory: / Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: detected binary path: /usr/bin/uwsgi-core Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: your processes number limit is 47680 Aug 09 18:59:35.398588 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: your memory page size is 4096 bytes Aug 09 18:59:35.399237 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: detected max file descriptor number: 2048 Aug 09 18:59:35.399237 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: lock engine: pthread robust mutexes Aug 09 18:59:35.399237 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: thunder lock: enabled Aug 09 18:59:35.399237 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 09 18:59:35.399237 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 09 18:59:35.439444 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: Python main interpreter initialized at 0x5566266779c0 Aug 09 18:59:35.439444 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: python threads support enabled Aug 09 18:59:35.439444 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: your server socket listen backlog is limited to 100 connections Aug 09 18:59:35.439444 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: your mercy for graceful operations on workers is 90 seconds Aug 09 18:59:35.439790 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: mapped 403077 bytes (393 KB) for 2 cores Aug 09 18:59:35.439867 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: *** Operational MODE: preforking *** Aug 09 18:59:35.440108 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: *** uWSGI is running in multiple interpreter mode *** Aug 09 18:59:35.440108 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: spawned uWSGI master process (pid: 80327) Aug 09 18:59:35.440309 dv-cinder-852654-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 09 18:59:35.440870 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: spawned uWSGI worker 1 (pid: 80338, cores: 1) Aug 09 18:59:35.445311 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: spawned uWSGI worker 2 (pid: 80339, cores: 1) Aug 09 18:59:35.445311 dv-cinder-852654-1-iscsi devstack@keystone.service[80327]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 09 18:59:40.944965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 18:59:40.944965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: from cryptography.utils import int_from_bytes Aug 09 18:59:40.945787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 18:59:40.945787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: from cryptography.utils import int_from_bytes Aug 09 18:59:40.946202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 18:59:40.946202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: from cryptography.utils import int_from_bytes Aug 09 18:59:40.946309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 18:59:40.946309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: from cryptography.utils import int_from_bytes Aug 09 18:59:40.994330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 18:59:40.994533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 18:59:40.994688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] command line args: [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 18:59:40.994952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 18:59:40.995039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 18:59:40.995120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ================================================================================ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 18:59:40.995203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 18:59:40.995366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] admin_token = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.995454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] command line args: [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 18:59:40.995608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_dir = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.995689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 18:59:40.995793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.995881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ================================================================================ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 18:59:40.995977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_source = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] admin_token = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] debug = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] config_dir = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] config_source = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] debug = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.996877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.997026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.997669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.997802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_config_append = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_dir = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.998917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_config_append = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_dir = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_file = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:40.999926 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000639 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.000927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.001988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.002083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.002231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.002313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.002521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.002716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.002804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.002922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] publish_errors = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.003028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.003115 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.003320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.003500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.003629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.003711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] publish_errors = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.003793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.003943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] standard_threads = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.004976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_journal = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] standard_threads = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_json = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_stderr = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_syslog = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_journal = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.005888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.006013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_json = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.006097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.006169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_stderr = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.006298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_syslog = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.006375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.006527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 18:59:41.006612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.006779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.006864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.007994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.008152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.008230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.008479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.008559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.008728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.008822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.008975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.009061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.009221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.009353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.009455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.009657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.009744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.009922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.010005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.010168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.010257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.010405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.010519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.010637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.010822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.010902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.011128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.011214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.011377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.011456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.011612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.011761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.011875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.012126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.012207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.012382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.012492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.012654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.013498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.013600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.013733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.013847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.013966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.014204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.014290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.014455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.014535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.014705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.014840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.014945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.015140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.015231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.015385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.015469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.015632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.015753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.015868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.016047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.016125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.016333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.016417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.016594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.016688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.016831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.016909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017824 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.017981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.018980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.019137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.019225 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.019371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.019457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.019601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.019678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.019831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.019937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.020065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.020179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.020279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.020373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.020542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.020660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.020792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.020922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.021988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.022139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.022222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.022385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.022472 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.022621 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.022704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.022851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.022932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.023080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.023164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.023315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.023397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.023556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.023638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.023785 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.023866 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.024988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.025068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.025228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.025315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.025462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.025541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.025696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.025779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.025928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.026951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.027092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.027171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.027326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.027412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.027559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.027639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.027791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.027871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.028022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.028102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.028263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.028505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.028593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.028730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.028894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.029019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.029134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.029253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.029487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.029571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.029720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.029957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.030049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.030189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.030364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.030444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.030652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.030788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.030900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.031137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.031223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.031396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.031631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.031719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.031884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.032028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.032109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.external = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.032336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.032487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.032653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.032886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.032971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.033117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.password = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.033348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.token = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.033425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.033586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.033795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.033876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.034051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.034221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.034302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.034514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.034679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.034769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.034988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.035114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.035224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.035471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.035579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.035719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.035966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.036099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.036231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.036428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.036587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.036725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.036891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.external = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.037020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.037175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.037304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.037430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.037652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.037873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.038000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.038166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.038292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.038420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.038559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.password = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.038685 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.038808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.token = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.038936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.039062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.039190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.039324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.039463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.039592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.039730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.039859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.039985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.040120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.040255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.040382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.040544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.040669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.040797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.040934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.041060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.041188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.041315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.041452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.041583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.041716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.041844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.041971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.042096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.042222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.042348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.042483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.042613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.042744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.042870 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.042997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.043123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.043273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.043434 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.043558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.043685 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.043762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.043834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.043918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.044942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.045975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.046055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.046228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.046310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.046462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.046542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.046707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.046786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.046940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.047968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.048114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.048192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.048347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.048450 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.048616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.048697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.048914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.049860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.050946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.051993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.052141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.052219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.052366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.052473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.052624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.052707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.052851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.052932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.053085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.053164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.053318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.053399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.053552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.053631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.053781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.053864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054811 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.054971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055742 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.055976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.056118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.056198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.056348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.056426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.056599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.056692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.056839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.056918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.057074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.057154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.057314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.057401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.057556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.057634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.057783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.057863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.058950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.059031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.059181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.059260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.059406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.059493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.060185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.060271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.060423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.060683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.060766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.060919 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.061001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.061420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.061501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.061671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.061751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.061915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.061998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.062148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.062228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.062377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.062459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.062610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.062692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.062842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.062924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.063077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.063435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.063435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.063587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.063659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.063736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.063858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.063934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.064962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] role.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.065924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] role.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.066083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.066165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.066327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.066407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.066560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.066639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.066809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.066890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067355 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067821 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.067974 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.068054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.068198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.068273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.068458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.068540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.068694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.068774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.068927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.069009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.069164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.069244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.069408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.069578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.069659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.069822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.069904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.070078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.070159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.070331 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.070412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.070562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.070640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.070807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.070887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071041 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.071995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.072150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.072229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.072386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.072494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.072644 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.072726 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.072896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.072976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.073150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.073237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.073394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.073473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.073633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.073712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.073869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.073957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.074098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.074177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.074332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.074410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.074569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.074657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.074808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.074888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.075045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.075124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.075970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.076181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.076323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.076487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.076618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.076761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.076902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.077032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.077161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.077289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.077416 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.077543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.077678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.077807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.077934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.078061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.078189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.078316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.078460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.078587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.078715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.078843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.078970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.079097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.079233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.079360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.079488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.079615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.079742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.079884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.080019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.080146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.080292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.080419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.080521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.080634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.080765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.080890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.081029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.081160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.081288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.081418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.081545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.081673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.081809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.081938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.082076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.082076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.082320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.082448 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.082584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.082712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.082840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.082968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.083096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.083224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.083360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.083490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.083619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.083759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.083888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 18:59:41.084684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.084684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.085271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.085271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.085536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.085536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.085783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.085991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.086224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.086457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.086689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.086929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.087160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.087397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.087627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.087870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.088100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.088340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.088590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.088826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.089056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 18:59:41.089223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 18:59:41.112975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.113210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.113437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.113747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.113978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.114203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.124036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.124655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.152890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.153103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.153188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.153524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.206207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.206474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.256642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.256869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.257263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.257500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.323192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.323418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.323614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.323706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.323841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.323929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.324062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.324147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.324274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.324357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.324509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.324726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.347314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.347452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.347533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.347664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.347833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.347915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.347990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.348101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.348196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.348319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.348403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.348578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.348663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.348840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.360261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.360789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.373692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.373898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.373987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374654 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.374984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.375067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.375196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 18:59:41.392332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.392506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.392601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 18:59:41.392756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 18:59:41.392914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.393052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.393135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 18:59:41.393254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 18:59:41.393378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.393517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.393641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.393774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.393898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.394029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.394157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.394272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.405646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.405754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.405921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.406006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.406188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.406287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.406469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.406552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.413355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.413482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.416393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.416570 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.416700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.416844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.416990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.417127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.417269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.417409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.417553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.417693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.417811 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.417958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.418092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.418227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.418371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.418499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.418642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.418786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.419442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.419568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.419728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.419841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.443685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.443810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.445188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.445280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.445481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.445565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.450047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.450142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.450367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.450451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.450651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.450738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.457341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.457439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.457575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.457675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.461568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.461660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.461859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.461943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.462140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.462225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.462420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.462504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.462691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.462774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.462960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.463046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.463228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.463311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.463493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.463585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.479240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.479517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.479611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.479843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.484043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.484478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.487169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.487461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.487624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.487921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.491813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.492302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.494968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.495499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.498916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.499213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.499299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.499506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.499590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.499798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.499891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.500076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.500160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.500362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.500473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.500679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.516292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.516602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.516870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.516953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.517129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.517241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.517515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.517779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.524023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.524700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.525596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.525874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.526258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.526537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.528757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.529042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.529326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.529491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.529621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.529780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.529894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.530073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.530374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.530655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.541282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.541566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.541846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.542113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.542198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.542389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.542481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.542665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.542749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.542939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.543031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.543216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.543492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.543779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.560534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.561846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.563741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.564859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.566758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.568073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.568326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.569694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.569832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.570129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.570417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.571165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.571460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.571744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.576844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.578439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.578834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.580465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.581372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.581662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.581946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.582223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.583369 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.583663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.583950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.584228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.590184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.590476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.590756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.592311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.592637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.592920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.597641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.597926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.598206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.598482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.598755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.599027 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.599300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.599568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.599813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 18:59:41.599953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.600101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.600235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.600386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.600530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.600689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.600795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.600964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.601096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.601239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.601322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.601515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.601798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.602177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.602460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.602734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.603000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.603283 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.603549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 18:59:41.635341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5566266779c0 pid: 80339 (default app) Aug 09 18:59:41.638617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5566266779c0 pid: 80338 (default app) Aug 09 18:59:41.638720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d4df6-0d5b-433d-8610-2f3a29ebd3fa None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:41.638960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d4df6-0d5b-433d-8610-2f3a29ebd3fa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:41.639230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d4df6-0d5b-433d-8610-2f3a29ebd3fa None None] PATH_INFO: `/v3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:41.643482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1/1] 10.222.0.70 () {50 vars in 791 bytes} [Tue Aug 9 18:59:41 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:59:49.427839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21023a-3157-4e07-9453-122fdca04856 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:49.428371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21023a-3157-4e07-9453-122fdca04856 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:49.428475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21023a-3157-4e07-9453-122fdca04856 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:49.432855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1/2] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:59:49.439249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:49.439676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:49.439767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:49.518157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: INFO dbcounter [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] Registered counter for database keystone Aug 09 18:59:49.542569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG oslo_db.sqlalchemy.engines [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80339) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 18:59:49.549357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writer thread running {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 18:59:49.724376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG passlib.handlers.bcrypt [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80339) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 18:59:49.724822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG passlib.handlers.bcrypt [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80339) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 18:59:49.779257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:49.780858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:49.922099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 2/3] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 18:59:50.073803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21023a-3157-4e07-9453-122fdca04856 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:50.143679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: INFO dbcounter [None req-1f21023a-3157-4e07-9453-122fdca04856 None None] Registered counter for database keystone Aug 09 18:59:50.165185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG oslo_db.sqlalchemy.engines [None req-1f21023a-3157-4e07-9453-122fdca04856 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80338) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 18:59:50.172066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writer thread running {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 18:59:50.290991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21023a-3157-4e07-9453-122fdca04856 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:50.296024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798c46a-d39e-4236-ab62-df19336bb3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:50.296678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798c46a-d39e-4236-ab62-df19336bb3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:50.296960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798c46a-d39e-4236-ab62-df19336bb3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:50.297236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798c46a-d39e-4236-ab62-df19336bb3e5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:50.306634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798c46a-d39e-4236-ab62-df19336bb3e5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:50.359870 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8798c46a-d39e-4236-ab62-df19336bb3e5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 18:59:50.360153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798c46a-d39e-4236-ab62-df19336bb3e5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:50.369817 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 2/4] 10.222.0.70 () {58 vars in 1324 bytes} [Tue Aug 9 18:59:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:59:50.376520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:50.391233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bff2ef-71ff-44ed-bcd5-6bab7ffeb850 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:50.395640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104eec85-2565-4268-bc0b-39a847e76d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:50.396792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eec85-2565-4268-bc0b-39a847e76d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:50.397075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eec85-2565-4268-bc0b-39a847e76d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:50.397345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eec85-2565-4268-bc0b-39a847e76d03 admin admin] PATH_INFO: `/v3/projects/de422168330743c9990b96b3dab6b6f7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:50.398770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104eec85-2565-4268-bc0b-39a847e76d03 admin admin] RBAC: Authorizing `identity:get_project(project_id=de422168330743c9990b96b3dab6b6f7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:50.449506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING keystone.common.rbac_enforcer.enforcer [None req-104eec85-2565-4268-bc0b-39a847e76d03 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 18:59:50.450029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104eec85-2565-4268-bc0b-39a847e76d03 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:50.454614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 3/5] 10.222.0.70 () {56 vars in 1207 bytes} [Tue Aug 9 18:59:50 2022] GET /identity/v3/projects/de422168330743c9990b96b3dab6b6f7 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:59:52.439913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5444d96-f3e8-453b-b8d3-be3f2197e334 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:52.440417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5444d96-f3e8-453b-b8d3-be3f2197e334 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:52.440543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5444d96-f3e8-453b-b8d3-be3f2197e334 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:52.444566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 3/6] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:59:52.450595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeac74-f606-49eb-9d40-9353e77d9ba8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:52.450997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeac74-f606-49eb-9d40-9353e77d9ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:52.451181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeac74-f606-49eb-9d40-9353e77d9ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:52.472085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1aaeac74-f606-49eb-9d40-9353e77d9ba8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:52.473641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1aaeac74-f606-49eb-9d40-9353e77d9ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:52.481516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 4/7] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 18:59:52.505635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5444d96-f3e8-453b-b8d3-be3f2197e334 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:52.518176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5444d96-f3e8-453b-b8d3-be3f2197e334 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:52.522346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29468bcc-6b3f-4cb9-8b45-a38e41a25420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:52.522933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29468bcc-6b3f-4cb9-8b45-a38e41a25420 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:52.523209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29468bcc-6b3f-4cb9-8b45-a38e41a25420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:52.523476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29468bcc-6b3f-4cb9-8b45-a38e41a25420 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:52.531044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29468bcc-6b3f-4cb9-8b45-a38e41a25420 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:52.532834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29468bcc-6b3f-4cb9-8b45-a38e41a25420 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:52.541478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 4/8] 10.222.0.70 () {58 vars in 1324 bytes} [Tue Aug 9 18:59:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:59:52.547660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeac74-f606-49eb-9d40-9353e77d9ba8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:52.556714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeac74-f606-49eb-9d40-9353e77d9ba8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:52.562014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be309be8-61d5-492c-ad9f-73bd97bf93e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:52.562632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be309be8-61d5-492c-ad9f-73bd97bf93e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:52.562909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be309be8-61d5-492c-ad9f-73bd97bf93e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:52.563188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be309be8-61d5-492c-ad9f-73bd97bf93e8 admin admin] PATH_INFO: `/v3/users/910b0103ce4a49babfe0b9390ae4fdd2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:52.564541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be309be8-61d5-492c-ad9f-73bd97bf93e8 admin admin] RBAC: Authorizing `identity:get_user(user_id=910b0103ce4a49babfe0b9390ae4fdd2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:52.566352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be309be8-61d5-492c-ad9f-73bd97bf93e8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:52.567845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 5/9] 10.222.0.70 () {56 vars in 1198 bytes} [Tue Aug 9 18:59:52 2022] GET /identity/v3/users/910b0103ce4a49babfe0b9390ae4fdd2 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:59:54.645543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eecfe1-8a43-49c2-a6cb-d47244a424f5 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:54.646051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eecfe1-8a43-49c2-a6cb-d47244a424f5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:54.646051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eecfe1-8a43-49c2-a6cb-d47244a424f5 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:54.647308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 5/10] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:59:54.652419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cca87-eb11-4c61-9c99-8493491d24d9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:54.652863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cca87-eb11-4c61-9c99-8493491d24d9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:54.653035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cca87-eb11-4c61-9c99-8493491d24d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:54.658634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47af0a-2ee4-43f5-adb9-745378a6a49c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:54.658915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47af0a-2ee4-43f5-adb9-745378a6a49c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:54.659188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47af0a-2ee4-43f5-adb9-745378a6a49c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:54.660427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 6/11] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:59:54.665528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:54.665971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:54.666249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:54.670886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b20cca87-eb11-4c61-9c99-8493491d24d9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:54.672318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b20cca87-eb11-4c61-9c99-8493491d24d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:54.711174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG passlib.handlers.bcrypt [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80338) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 18:59:54.711610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG passlib.handlers.bcrypt [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80338) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 18:59:54.753050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 6/12] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:59:54.757211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:54.758851 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:54.776076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cca87-eb11-4c61-9c99-8493491d24d9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:54.800983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:54.860509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 7/13] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:59:54.881726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:54.882241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cca87-eb11-4c61-9c99-8493491d24d9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:54.919309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32168d-155e-462b-9798-d3fcf6e8c152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:54.919950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32168d-155e-462b-9798-d3fcf6e8c152 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:54.920231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32168d-155e-462b-9798-d3fcf6e8c152 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:54.920617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32168d-155e-462b-9798-d3fcf6e8c152 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:54.931836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32168d-155e-462b-9798-d3fcf6e8c152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6d32168d-155e-462b-9798-d3fcf6e8c152 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:59:54.940960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:59:54.941925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 18:59:54.942720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 7/14] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 18:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:59:54.948238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32168d-155e-462b-9798-d3fcf6e8c152 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:54.987629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafc952-b63a-4e65-8eab-b802165586e7 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:54.990176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:55.026316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b0b2e-a2c0-4e74-8511-4db842276681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:55.027183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b0b2e-a2c0-4e74-8511-4db842276681 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:55.027467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b0b2e-a2c0-4e74-8511-4db842276681 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:55.027738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b0b2e-a2c0-4e74-8511-4db842276681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:55.033199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:55.051282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32168d-155e-462b-9798-d3fcf6e8c152 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:55.085960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:55.086561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:55.086833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:55.087099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:55.087757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:59:55.088053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:55.089466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:55.097679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b0b2e-a2c0-4e74-8511-4db842276681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:55.099186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b0b2e-a2c0-4e74-8511-4db842276681 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:55.109273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 8/15] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 18:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:59:55.115094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:55.198668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 8/16] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 18:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 18:59:55.204845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b0b2e-a2c0-4e74-8511-4db842276681 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:55.208072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:55.314783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b0b2e-a2c0-4e74-8511-4db842276681 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:55.323819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3dc66-3812-41be-8616-c728e119304e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:55.349470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:55.350338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:55.350615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:55.350879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:55.358588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2eee7-91dd-4bef-80f2-27b578c6c8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:55.359247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2eee7-91dd-4bef-80f2-27b578c6c8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:55.359529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2eee7-91dd-4bef-80f2-27b578c6c8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:55.359803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2eee7-91dd-4bef-80f2-27b578c6c8c9 None admin] PATH_INFO: `/v3/users/910b0103ce4a49babfe0b9390ae4fdd2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:55.360961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2eee7-91dd-4bef-80f2-27b578c6c8c9 None admin] RBAC: Authorizing `identity:get_user(user_id=910b0103ce4a49babfe0b9390ae4fdd2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:55.362420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2eee7-91dd-4bef-80f2-27b578c6c8c9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:55.363976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 9/17] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 18:59:55 2022] GET /identity/v3/users/910b0103ce4a49babfe0b9390ae4fdd2 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:59:55.369382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2eee7-91dd-4bef-80f2-27b578c6c8c9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:55.370557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:55.372725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:55.389674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:59:55.391802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:59:55.393062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:59:55.393855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:59:55.393855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:59:55.393855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 18:59:55.393855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 9/18] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 18:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:59:55.398954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:55.455278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:55.472017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2eee7-91dd-4bef-80f2-27b578c6c8c9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:55.500214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd30f-4343-4b45-a990-dc2660b92194 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:55.506082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b51408-d1b0-47a9-b47c-61a2354116eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:55.506718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51408-d1b0-47a9-b47c-61a2354116eb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:55.506983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51408-d1b0-47a9-b47c-61a2354116eb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:55.507279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51408-d1b0-47a9-b47c-61a2354116eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:55.508365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b51408-d1b0-47a9-b47c-61a2354116eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:55.509831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b51408-d1b0-47a9-b47c-61a2354116eb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:55.511360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 10/19] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 18:59:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:59:55.518145 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b51408-d1b0-47a9-b47c-61a2354116eb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:55.536202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4ae9c-6f13-42f5-8e9d-daf7e95b7bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:55.536852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4ae9c-6f13-42f5-8e9d-daf7e95b7bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:55.537133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4ae9c-6f13-42f5-8e9d-daf7e95b7bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:55.537407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4ae9c-6f13-42f5-8e9d-daf7e95b7bf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:55.538052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4ae9c-6f13-42f5-8e9d-daf7e95b7bf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:59:55.538359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4ae9c-6f13-42f5-8e9d-daf7e95b7bf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:55.539792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4ae9c-6f13-42f5-8e9d-daf7e95b7bf9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:55.557319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 10/20] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 18:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:59:55.603247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:55.626092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b51408-d1b0-47a9-b47c-61a2354116eb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:55.660637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c6b0b-2454-4881-9fb1-123ccfd60e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:55.661330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c6b0b-2454-4881-9fb1-123ccfd60e30 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:55.661514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c6b0b-2454-4881-9fb1-123ccfd60e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:55.661783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c6b0b-2454-4881-9fb1-123ccfd60e30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:55.662526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c6b0b-2454-4881-9fb1-123ccfd60e30 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.domain.id=default, user.id=910b0103ce4a49babfe0b9390ae4fdd2) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:59:55.662829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c6b0b-2454-4881-9fb1-123ccfd60e30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:55.664309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c6b0b-2454-4881-9fb1-123ccfd60e30 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:55.685007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 11/21] 10.222.0.70 () {56 vars in 1322 bytes} [Tue Aug 9 18:59:55 2022] GET /identity/v3/role_assignments?user.id=910b0103ce4a49babfe0b9390ae4fdd2&scope.domain.id=default&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:59:57.923335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc9ae0-8cf9-4eab-baaf-11912074dff3 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:57.924349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc9ae0-8cf9-4eab-baaf-11912074dff3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:57.924863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc9ae0-8cf9-4eab-baaf-11912074dff3 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:57.926670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 11/22] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:59:57.931747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92908dd-002a-4233-b272-174a92eb501e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:57.932175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92908dd-002a-4233-b272-174a92eb501e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:57.932290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92908dd-002a-4233-b272-174a92eb501e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:57.958641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f92908dd-002a-4233-b272-174a92eb501e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:57.960180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f92908dd-002a-4233-b272-174a92eb501e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:58.049562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 12/23] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:59:58.093105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc9ae0-8cf9-4eab-baaf-11912074dff3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:58.182259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd298-0f4a-4806-833f-e017c5c5a89f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.182459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd298-0f4a-4806-833f-e017c5c5a89f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.183077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefd298-0f4a-4806-833f-e017c5c5a89f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.186106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 13/24] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:59:58.193047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bcc88-f197-4146-a510-530afde0c8b5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.193591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bcc88-f197-4146-a510-530afde0c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.194231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bcc88-f197-4146-a510-530afde0c8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.241839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f92bcc88-f197-4146-a510-530afde0c8b5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:58.243821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f92bcc88-f197-4146-a510-530afde0c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:58.286946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc9ae0-8cf9-4eab-baaf-11912074dff3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:58.316905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:58.358642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfaa34-eec7-4d85-8685-acec08efd095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:58.359263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfaa34-eec7-4d85-8685-acec08efd095 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.359538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfaa34-eec7-4d85-8685-acec08efd095 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.359809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfaa34-eec7-4d85-8685-acec08efd095 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.360500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfaa34-eec7-4d85-8685-acec08efd095 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:58.361887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfaa34-eec7-4d85-8685-acec08efd095 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:58.386825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 14/25] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 18:59:58.394650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ff19a-fd60-4015-ac22-e325bfa996c7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.396124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ff19a-fd60-4015-ac22-e325bfa996c7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.397374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ff19a-fd60-4015-ac22-e325bfa996c7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.402562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 15/26] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:58 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:59:58.406161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847902d1-9bdb-41cd-83b4-5319f1808015 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.406591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847902d1-9bdb-41cd-83b4-5319f1808015 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.407077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847902d1-9bdb-41cd-83b4-5319f1808015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.432886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:58.445164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-847902d1-9bdb-41cd-83b4-5319f1808015 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:58.448705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-847902d1-9bdb-41cd-83b4-5319f1808015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:58.598327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 16/27] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:59:58.600057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847902d1-9bdb-41cd-83b4-5319f1808015 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:58.712558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847902d1-9bdb-41cd-83b4-5319f1808015 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:58.716792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 12/28] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Aug 9 18:59:58 2022] POST /identity/v3/roles => generated 219 bytes in 624 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:58.718243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330318-5072-4594-9cc7-fbc787dcba95 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.718528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330318-5072-4594-9cc7-fbc787dcba95 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.718805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330318-5072-4594-9cc7-fbc787dcba95 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.720481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 13/29] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:59:58.721237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330318-5072-4594-9cc7-fbc787dcba95 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:58.773921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c625f-6f5d-414a-8e67-3fb52401e3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:58.774964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c625f-6f5d-414a-8e67-3fb52401e3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.775448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c625f-6f5d-414a-8e67-3fb52401e3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.775937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c625f-6f5d-414a-8e67-3fb52401e3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.783218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:58.802137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c625f-6f5d-414a-8e67-3fb52401e3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:58.805643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c625f-6f5d-414a-8e67-3fb52401e3a8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-774c625f-6f5d-414a-8e67-3fb52401e3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 18:59:58.843526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:59:58.844414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 18:59:58.845226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 18:59:58.845226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 18:59:58.845226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 18:59:58.845226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 17/30] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 18:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 18:59:58.845986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1322ac-0be6-4377-8680-b3facca86cb4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.846269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1322ac-0be6-4377-8680-b3facca86cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.846546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1322ac-0be6-4377-8680-b3facca86cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.859510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330318-5072-4594-9cc7-fbc787dcba95 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:58.889819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ee1322ac-0be6-4377-8680-b3facca86cb4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:58.891166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ee1322ac-0be6-4377-8680-b3facca86cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:58.914487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f4b1-3645-4c32-898b-f12cd1010781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:58.915681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f4b1-3645-4c32-898b-f12cd1010781 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:58.916720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f4b1-3645-4c32-898b-f12cd1010781 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:58.917480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f4b1-3645-4c32-898b-f12cd1010781 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:58.919283 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54f4b1-3645-4c32-898b-f12cd1010781 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:58.921493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54f4b1-3645-4c32-898b-f12cd1010781 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:58.943770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 14/31] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Aug 9 18:59:58 2022] POST /identity/v3/roles => generated 225 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:58.945459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f4b1-3645-4c32-898b-f12cd1010781 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:58.989328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 18/32] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:59:59.014816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:59.019063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1322ac-0be6-4377-8680-b3facca86cb4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:59.156234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1322ac-0be6-4377-8680-b3facca86cb4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:59.170371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f4b1-3645-4c32-898b-f12cd1010781 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:59.176603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:59.202789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616db46a-5fdb-4c6a-9113-95e85977706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:59.202789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616db46a-5fdb-4c6a-9113-95e85977706d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.202789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616db46a-5fdb-4c6a-9113-95e85977706d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.202789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616db46a-5fdb-4c6a-9113-95e85977706d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.221999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616db46a-5fdb-4c6a-9113-95e85977706d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 18:59:59.227403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-616db46a-5fdb-4c6a-9113-95e85977706d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 18:59:59.227629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 18:59:59.228374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 18:59:59.229141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 19/33] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 18:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 18:59:59.233580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.234118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.234837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.235430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616db46a-5fdb-4c6a-9113-95e85977706d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:59.236056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:59:59.236660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:59.238330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:59.253654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 15/34] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 18:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:59:59.259813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:59.377417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616db46a-5fdb-4c6a-9113-95e85977706d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:59.381510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:59.390839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fcbf6-cfa7-42dd-82be-db1bebbb0e5e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:59.416340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:59.417343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.417808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.418281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.419745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:59:59.420276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:59.421902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:59.433624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 20/35] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 18:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:59:59.440088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e73a71-76f6-4b74-8ea0-94566aacb9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:59.441235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:59.442126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e73a71-76f6-4b74-8ea0-94566aacb9ae None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.442407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e73a71-76f6-4b74-8ea0-94566aacb9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.442681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e73a71-76f6-4b74-8ea0-94566aacb9ae None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.443599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e73a71-76f6-4b74-8ea0-94566aacb9ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:59.445537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e73a71-76f6-4b74-8ea0-94566aacb9ae None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:59.488236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 16/36] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 18:59:59 2022] POST /identity/v3/projects => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:59:59.491318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:59.511977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a137-5f55-4da2-88eb-8f057a3886de None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.512730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a137-5f55-4da2-88eb-8f057a3886de None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.513224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a137-5f55-4da2-88eb-8f057a3886de None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.514899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 17/37] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 18:59:59.520928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658e33-a4b2-4494-9f4d-3cf70d6c5246 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.521209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658e33-a4b2-4494-9f4d-3cf70d6c5246 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.521546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658e33-a4b2-4494-9f4d-3cf70d6c5246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.542205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2b658e33-a4b2-4494-9f4d-3cf70d6c5246 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:59.544678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2b658e33-a4b2-4494-9f4d-3cf70d6c5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:59.564979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c82331-75e2-448a-96f9-070e29cbb317 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:59.605018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2fc9-6a2a-4854-9654-9eb6d45aa38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:59.605782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2fc9-6a2a-4854-9654-9eb6d45aa38d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.606648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2fc9-6a2a-4854-9654-9eb6d45aa38d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.607105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2fc9-6a2a-4854-9654-9eb6d45aa38d None admin] PATH_INFO: `/v3/users/910b0103ce4a49babfe0b9390ae4fdd2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.608331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2fc9-6a2a-4854-9654-9eb6d45aa38d None admin] RBAC: Authorizing `identity:get_user(user_id=910b0103ce4a49babfe0b9390ae4fdd2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:59.609956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2fc9-6a2a-4854-9654-9eb6d45aa38d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:59.611825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 21/38] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 18:59:59 2022] GET /identity/v3/users/910b0103ce4a49babfe0b9390ae4fdd2 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:59:59.617367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2fc9-6a2a-4854-9654-9eb6d45aa38d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:59.634194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 18/39] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:59:59.659270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b658e33-a4b2-4494-9f4d-3cf70d6c5246 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:59.749507 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:59.750116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2fc9-6a2a-4854-9654-9eb6d45aa38d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:59.758168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:59:59.772793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b658e33-a4b2-4494-9f4d-3cf70d6c5246 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:59.785923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a08f9-bacf-4914-881d-d8b500a0b4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:59.786547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a08f9-bacf-4914-881d-d8b500a0b4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.786821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a08f9-bacf-4914-881d-d8b500a0b4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.787090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a08f9-bacf-4914-881d-d8b500a0b4b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.788189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a08f9-bacf-4914-881d-d8b500a0b4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:59.789659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a08f9-bacf-4914-881d-d8b500a0b4b0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:59.791197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 22/40] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 18:59:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:59:59.792292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fac4a-7495-4d5c-a4c7-0714972bf6dc None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.792590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fac4a-7495-4d5c-a4c7-0714972bf6dc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.792858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fac4a-7495-4d5c-a4c7-0714972bf6dc None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.794933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 23/41] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 18:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:59:59.797390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fac4a-7495-4d5c-a4c7-0714972bf6dc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:59.811443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ab777-0346-483a-9bfe-1980623a65fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:59.812054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab777-0346-483a-9bfe-1980623a65fe None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.812330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab777-0346-483a-9bfe-1980623a65fe None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.812631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab777-0346-483a-9bfe-1980623a65fe None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.813294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ab777-0346-483a-9bfe-1980623a65fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:59.814681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ab777-0346-483a-9bfe-1980623a65fe None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:59:59.830512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 19/42] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Aug 9 18:59:59 2022] POST /identity/v3/roles => generated 223 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:59:59.831722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be579faf-1e89-40ca-be85-0b58889c8690 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.832109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be579faf-1e89-40ca-be85-0b58889c8690 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.832389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be579faf-1e89-40ca-be85-0b58889c8690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.851721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-be579faf-1e89-40ca-be85-0b58889c8690 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:59:59.853022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-be579faf-1e89-40ca-be85-0b58889c8690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:59:59.916345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fac4a-7495-4d5c-a4c7-0714972bf6dc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:59:59.930373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 20/43] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 18:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 18:59:59.954519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be579faf-1e89-40ca-be85-0b58889c8690 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:59:59.956703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce28c84-cc60-44df-9e6f-d25fb739a616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:59:59.957615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce28c84-cc60-44df-9e6f-d25fb739a616 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:59:59.957890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce28c84-cc60-44df-9e6f-d25fb739a616 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:59:59.958229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce28c84-cc60-44df-9e6f-d25fb739a616 None admin] PATH_INFO: `/v3/domains/default/users/910b0103ce4a49babfe0b9390ae4fdd2/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:59:59.960790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce28c84-cc60-44df-9e6f-d25fb739a616 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=910b0103ce4a49babfe0b9390ae4fdd2, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:59:59.962347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce28c84-cc60-44df-9e6f-d25fb739a616 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:00.034112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:00.069899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be579faf-1e89-40ca-be85-0b58889c8690 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:00.084113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 24/44] 10.222.0.70 () {58 vars in 1361 bytes} [Tue Aug 9 18:59:59 2022] PUT /identity/v3/domains/default/users/910b0103ce4a49babfe0b9390ae4fdd2/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:00:00.107383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c63c9-2e8b-4be9-b8ee-05f7b57b1aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:00.108042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c63c9-2e8b-4be9-b8ee-05f7b57b1aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:00.108321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c63c9-2e8b-4be9-b8ee-05f7b57b1aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:00.108624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c63c9-2e8b-4be9-b8ee-05f7b57b1aa5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:00.109847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c63c9-2e8b-4be9-b8ee-05f7b57b1aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:00.111338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c63c9-2e8b-4be9-b8ee-05f7b57b1aa5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:00.113006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 21/45] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 18:59:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:00.119375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce28c84-cc60-44df-9e6f-d25fb739a616 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:00.157548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:00.230215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce28c84-cc60-44df-9e6f-d25fb739a616 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:00.267196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d4dfb-7925-46bc-86e0-4b925da9b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:00.267804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4dfb-7925-46bc-86e0-4b925da9b928 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:00.268083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4dfb-7925-46bc-86e0-4b925da9b928 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:00.268353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4dfb-7925-46bc-86e0-4b925da9b928 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:00.269075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d4dfb-7925-46bc-86e0-4b925da9b928 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:00.270714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d4dfb-7925-46bc-86e0-4b925da9b928 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:00.303286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 25/46] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:00:00 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:00:02.162468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa29fe-48f1-4883-942b-cafd459b2081 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.163000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa29fe-48f1-4883-942b-cafd459b2081 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.163000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa29fe-48f1-4883-942b-cafd459b2081 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.164335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 22/47] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:02.170256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b2399-fd7b-4e4f-bb4a-97fabea7f79d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.170702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b2399-fd7b-4e4f-bb4a-97fabea7f79d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.170791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b2399-fd7b-4e4f-bb4a-97fabea7f79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.189049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ef8b2399-fd7b-4e4f-bb4a-97fabea7f79d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:02.190552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ef8b2399-fd7b-4e4f-bb4a-97fabea7f79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:02.256004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 26/48] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:02.279940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa29fe-48f1-4883-942b-cafd459b2081 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:02.348189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f5269-7c7b-4418-8aa3-004046a9d8e3 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.348503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f5269-7c7b-4418-8aa3-004046a9d8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.348791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f5269-7c7b-4418-8aa3-004046a9d8e3 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.350135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 27/49] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:02.355088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0d693-b368-47fe-bfc2-0b6f181e3b2a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.355373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0d693-b368-47fe-bfc2-0b6f181e3b2a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.355651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0d693-b368-47fe-bfc2-0b6f181e3b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.374700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-11c0d693-b368-47fe-bfc2-0b6f181e3b2a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:02.376458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-11c0d693-b368-47fe-bfc2-0b6f181e3b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:02.388911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa29fe-48f1-4883-942b-cafd459b2081 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:02.391586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:02.425090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9f18-e966-4fda-b179-2c245642e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:02.425652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9f18-e966-4fda-b179-2c245642e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.425935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9f18-e966-4fda-b179-2c245642e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.426215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9f18-e966-4fda-b179-2c245642e2c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.430028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:02.440008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf9f18-e966-4fda-b179-2c245642e2c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-dacf9f18-e966-4fda-b179-2c245642e2c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:02.445973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:02.447133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:02.447847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 23/50] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:02.453141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 28/51] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:02.455161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9f18-e966-4fda-b179-2c245642e2c4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:02.475992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0d693-b368-47fe-bfc2-0b6f181e3b2a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:02.557087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9f18-e966-4fda-b179-2c245642e2c4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:02.577082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0d693-b368-47fe-bfc2-0b6f181e3b2a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:02.592288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:02.592922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.593204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.593474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.594149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:02.594451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:02.595826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:02.609545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 24/52] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:02.610975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257697b-2f93-42ed-ac3c-d71fda35f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:02.611617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257697b-2f93-42ed-ac3c-d71fda35f2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.611900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257697b-2f93-42ed-ac3c-d71fda35f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.612175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257697b-2f93-42ed-ac3c-d71fda35f2d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.613323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257697b-2f93-42ed-ac3c-d71fda35f2d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:02.614778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257697b-2f93-42ed-ac3c-d71fda35f2d2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:02.615024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:02.616328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 29/53] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:02.622238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257697b-2f93-42ed-ac3c-d71fda35f2d2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:02.637115 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:02.677439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:02.719581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257697b-2f93-42ed-ac3c-d71fda35f2d2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:02.720280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e64085-beb3-4152-b573-b975da5be6fc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:02.753208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a7c82-b845-49c7-a407-e5ea6f02f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:02.753538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97433c-689d-4fed-81a9-ffb9c37774f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:02.753812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a7c82-b845-49c7-a407-e5ea6f02f8c1 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.754092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a7c82-b845-49c7-a407-e5ea6f02f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.754197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97433c-689d-4fed-81a9-ffb9c37774f7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.754368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a7c82-b845-49c7-a407-e5ea6f02f8c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.754458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97433c-689d-4fed-81a9-ffb9c37774f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.754682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97433c-689d-4fed-81a9-ffb9c37774f7 None admin] PATH_INFO: `/v3/users/910b0103ce4a49babfe0b9390ae4fdd2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.755011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a7c82-b845-49c7-a407-e5ea6f02f8c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:02.756065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97433c-689d-4fed-81a9-ffb9c37774f7 None admin] RBAC: Authorizing `identity:get_user(user_id=910b0103ce4a49babfe0b9390ae4fdd2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:02.756538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a7c82-b845-49c7-a407-e5ea6f02f8c1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:02.757524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97433c-689d-4fed-81a9-ffb9c37774f7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:02.759073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 25/54] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:02 2022] GET /identity/v3/users/910b0103ce4a49babfe0b9390ae4fdd2 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:02.764736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97433c-689d-4fed-81a9-ffb9c37774f7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:02.784977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 30/55] 10.222.0.70 () {60 vars in 1140 bytes} [Tue Aug 9 19:00:02 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:00:02.872403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97433c-689d-4fed-81a9-ffb9c37774f7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:02.899284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:02.909126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f6f6b-6bce-4b85-851b-e6bda9f83666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:02.909772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6f6b-6bce-4b85-851b-e6bda9f83666 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:02.910059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6f6b-6bce-4b85-851b-e6bda9f83666 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:02.910339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6f6b-6bce-4b85-851b-e6bda9f83666 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:02.911493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f6f6b-6bce-4b85-851b-e6bda9f83666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:02.912990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f6f6b-6bce-4b85-851b-e6bda9f83666 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:02.914603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 26/56] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:02.921624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a7c82-b845-49c7-a407-e5ea6f02f8c1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:02.990417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:03.025913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a7c82-b845-49c7-a407-e5ea6f02f8c1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:03.072838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dffa39-afb4-47bc-ad0b-362285d55353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:03.073559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dffa39-afb4-47bc-ad0b-362285d55353 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:03.073852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dffa39-afb4-47bc-ad0b-362285d55353 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:03.074204 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dffa39-afb4-47bc-ad0b-362285d55353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:03.076239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dffa39-afb4-47bc-ad0b-362285d55353 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.domain.id=default, user.id=910b0103ce4a49babfe0b9390ae4fdd2) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:03.076592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dffa39-afb4-47bc-ad0b-362285d55353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:03.078079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dffa39-afb4-47bc-ad0b-362285d55353 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:03.084484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 31/57] 10.222.0.70 () {56 vars in 1322 bytes} [Tue Aug 9 19:00:02 2022] GET /identity/v3/role_assignments?user.id=910b0103ce4a49babfe0b9390ae4fdd2&scope.domain.id=default&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 530 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:04.845434 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53be3f-b346-404d-a3a8-60e7d672cbc1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:04.845951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53be3f-b346-404d-a3a8-60e7d672cbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:04.845951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53be3f-b346-404d-a3a8-60e7d672cbc1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:04.847305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 27/58] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:04.852464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1ca6-ad8d-4367-b910-d588594da4ff None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:04.852848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1ca6-ad8d-4367-b910-d588594da4ff None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:04.853027 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f1ca6-ad8d-4367-b910-d588594da4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:04.871317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5a2f1ca6-ad8d-4367-b910-d588594da4ff None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:04.873092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5a2f1ca6-ad8d-4367-b910-d588594da4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:04.938829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 32/59] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:04.963542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53be3f-b346-404d-a3a8-60e7d672cbc1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:05.064009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53be3f-b346-404d-a3a8-60e7d672cbc1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:05.113130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975fc3b-b3f9-499e-977e-48a30b99344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:05.113721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975fc3b-b3f9-499e-977e-48a30b99344a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:05.114048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975fc3b-b3f9-499e-977e-48a30b99344a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:05.114323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975fc3b-b3f9-499e-977e-48a30b99344a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:05.115425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975fc3b-b3f9-499e-977e-48a30b99344a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:05.117156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975fc3b-b3f9-499e-977e-48a30b99344a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:05.119123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 28/60] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:05.126896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1ca6-ad8d-4367-b910-d588594da4ff None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:05.187522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:05.227021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f1ca6-ad8d-4367-b910-d588594da4ff None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:05.259296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76fdb4-43f0-4839-96a2-ecacc0aa9387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:05.259895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76fdb4-43f0-4839-96a2-ecacc0aa9387 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:05.260172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76fdb4-43f0-4839-96a2-ecacc0aa9387 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:05.260466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76fdb4-43f0-4839-96a2-ecacc0aa9387 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:05.261104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76fdb4-43f0-4839-96a2-ecacc0aa9387 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:05.263122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76fdb4-43f0-4839-96a2-ecacc0aa9387 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:05.287810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 33/61] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:00:05 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:08.519405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000a3c-9e7b-4bbb-a9a9-5e151745526a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:08.520415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000a3c-9e7b-4bbb-a9a9-5e151745526a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:08.520927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000a3c-9e7b-4bbb-a9a9-5e151745526a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:08.522793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 29/62] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:08.528149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f2f6b-cfe8-4355-bcbb-b9ba31ecf24b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:08.529141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f2f6b-cfe8-4355-bcbb-b9ba31ecf24b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:08.529633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f2f6b-cfe8-4355-bcbb-b9ba31ecf24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:08.553498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f85f2f6b-cfe8-4355-bcbb-b9ba31ecf24b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:08.554707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f85f2f6b-cfe8-4355-bcbb-b9ba31ecf24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:08.576305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd3f5-6082-460b-a7e0-a473cfebd33d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:08.576978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd3f5-6082-460b-a7e0-a473cfebd33d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:08.577478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd3f5-6082-460b-a7e0-a473cfebd33d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:08.578899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 30/63] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:08.591590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081298b4-61d1-4916-a9ab-b0ad13284169 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:08.592484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081298b4-61d1-4916-a9ab-b0ad13284169 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:08.592955 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081298b4-61d1-4916-a9ab-b0ad13284169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:08.614710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-081298b4-61d1-4916-a9ab-b0ad13284169 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:08.615913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-081298b4-61d1-4916-a9ab-b0ad13284169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:08.630548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 34/64] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:08.632142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9ace-e933-4af9-a6c9-9619fa8d1ee1 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:08.632632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9ace-e933-4af9-a6c9-9619fa8d1ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:08.633082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9ace-e933-4af9-a6c9-9619fa8d1ee1 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:08.634504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 35/65] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:08.653315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199511d-fcc1-49cd-b5c4-78c31e04e744 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:08.653596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199511d-fcc1-49cd-b5c4-78c31e04e744 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:08.653867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199511d-fcc1-49cd-b5c4-78c31e04e744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:08.675827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f199511d-fcc1-49cd-b5c4-78c31e04e744 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:08.677293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f199511d-fcc1-49cd-b5c4-78c31e04e744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:08.698216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:08.714227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 31/66] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:08.716350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081298b4-61d1-4916-a9ab-b0ad13284169 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:08.730749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:08.768724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 36/67] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:08.769570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199511d-fcc1-49cd-b5c4-78c31e04e744 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:08.855408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081298b4-61d1-4916-a9ab-b0ad13284169 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:08.893916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b73ae-3de0-409a-beb6-4fc1f7086078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:08.894794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b73ae-3de0-409a-beb6-4fc1f7086078 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:08.895218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b73ae-3de0-409a-beb6-4fc1f7086078 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:08.895842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b73ae-3de0-409a-beb6-4fc1f7086078 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:08.897658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199511d-fcc1-49cd-b5c4-78c31e04e744 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:08.907812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b73ae-3de0-409a-beb6-4fc1f7086078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-9e0b73ae-3de0-409a-beb6-4fc1f7086078 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:08.912954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:08.913876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:08.914619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:08.916845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 32/68] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:08.923276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b73ae-3de0-409a-beb6-4fc1f7086078 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:08.936012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d35fcd-e44a-4a1f-919c-99362c8b84d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:08.936330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d35fcd-e44a-4a1f-919c-99362c8b84d0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:08.936790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d35fcd-e44a-4a1f-919c-99362c8b84d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:08.937222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d35fcd-e44a-4a1f-919c-99362c8b84d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:08.948183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d35fcd-e44a-4a1f-919c-99362c8b84d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f4d35fcd-e44a-4a1f-919c-99362c8b84d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:08.953304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:08.954132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:08.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 37/69] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:08.955481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d35fcd-e44a-4a1f-919c-99362c8b84d0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:08.986038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:08.997034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.040520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b73ae-3de0-409a-beb6-4fc1f7086078 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.072103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d35fcd-e44a-4a1f-919c-99362c8b84d0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.075484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9df731-196d-4315-9480-6c7d58c6501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.075691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9df731-196d-4315-9480-6c7d58c6501f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.075926 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9df731-196d-4315-9480-6c7d58c6501f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.076196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9df731-196d-4315-9480-6c7d58c6501f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.086417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9df731-196d-4315-9480-6c7d58c6501f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-fa9df731-196d-4315-9480-6c7d58c6501f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:09.091700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:09.092854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 33/70] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:09.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7cef9-84e2-413b-9eb6-6257082a627f None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.099236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7cef9-84e2-413b-9eb6-6257082a627f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.099236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7cef9-84e2-413b-9eb6-6257082a627f None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.099236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 34/71] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:09.099236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7cef9-84e2-413b-9eb6-6257082a627f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.120726 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.121405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.121708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.121950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.122602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:09.122907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:09.124303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:09.137927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 38/72] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:09.137927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.211305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7cef9-84e2-413b-9eb6-6257082a627f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.225241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.233905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.237956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599ae96-c83a-413b-9c24-28860fd78fe4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.248086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e07a28-35d6-48c1-b291-5bfb73b79e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.248750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07a28-35d6-48c1-b291-5bfb73b79e8a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.249032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07a28-35d6-48c1-b291-5bfb73b79e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.249310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e07a28-35d6-48c1-b291-5bfb73b79e8a None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.249973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e07a28-35d6-48c1-b291-5bfb73b79e8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:09.250282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e07a28-35d6-48c1-b291-5bfb73b79e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:09.251675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e07a28-35d6-48c1-b291-5bfb73b79e8a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:09.259608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 35/73] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:09.260788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a5742-bd67-4ad3-9e98-74a277587c93 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.261073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a5742-bd67-4ad3-9e98-74a277587c93 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.261350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a5742-bd67-4ad3-9e98-74a277587c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.275562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.276168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.276463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.276747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.277410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:09.278263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:09.279627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:09.279895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ba2a5742-bd67-4ad3-9e98-74a277587c93 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:09.281127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ba2a5742-bd67-4ad3-9e98-74a277587c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:09.287463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 39/74] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:09.291885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.349304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 36/75] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:09.350552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c4b13-176b-4648-97e9-2e010e077cd1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.350837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c4b13-176b-4648-97e9-2e010e077cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.351114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c4b13-176b-4648-97e9-2e010e077cd1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.352352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 37/76] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:09.353070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c4b13-176b-4648-97e9-2e010e077cd1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.399414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed71ab-976e-496d-9240-3db8db075342 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.432910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35eba-4c67-4ff7-a091-bfe0de97779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.433538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35eba-4c67-4ff7-a091-bfe0de97779a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.433936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35eba-4c67-4ff7-a091-bfe0de97779a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.434262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35eba-4c67-4ff7-a091-bfe0de97779a None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.447700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a35eba-4c67-4ff7-a091-bfe0de97779a None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:09.449181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a35eba-4c67-4ff7-a091-bfe0de97779a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:09.450819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 40/77] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:09.451571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35eba-4c67-4ff7-a091-bfe0de97779a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.462810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c4b13-176b-4648-97e9-2e010e077cd1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.465626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.471068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.503912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1745b-f606-4a50-bf49-906d74257a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.504162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1745b-f606-4a50-bf49-906d74257a09 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.504409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1745b-f606-4a50-bf49-906d74257a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.504710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1745b-f606-4a50-bf49-906d74257a09 None admin] PATH_INFO: `/v3/users/910b0103ce4a49babfe0b9390ae4fdd2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.505868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1745b-f606-4a50-bf49-906d74257a09 None admin] RBAC: Authorizing `identity:get_user(user_id=910b0103ce4a49babfe0b9390ae4fdd2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:09.507333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1745b-f606-4a50-bf49-906d74257a09 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:09.508947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 38/78] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/users/910b0103ce4a49babfe0b9390ae4fdd2 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:09.510150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa56197-912b-40d6-adc4-131e18ffbfb5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.510438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa56197-912b-40d6-adc4-131e18ffbfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.510718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa56197-912b-40d6-adc4-131e18ffbfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.529535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9fa56197-912b-40d6-adc4-131e18ffbfb5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:09.530722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9fa56197-912b-40d6-adc4-131e18ffbfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:09.560247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35eba-4c67-4ff7-a091-bfe0de97779a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.594425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22257e8b-37ac-42d3-be65-ceaf7a83e4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.595048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22257e8b-37ac-42d3-be65-ceaf7a83e4ee None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.595320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22257e8b-37ac-42d3-be65-ceaf7a83e4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.595589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22257e8b-37ac-42d3-be65-ceaf7a83e4ee None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.596718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22257e8b-37ac-42d3-be65-ceaf7a83e4ee None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:09.598149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22257e8b-37ac-42d3-be65-ceaf7a83e4ee None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:09.598275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 39/79] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:09.599112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa56197-912b-40d6-adc4-131e18ffbfb5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.599978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 41/80] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:09.600752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22257e8b-37ac-42d3-be65-ceaf7a83e4ee None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.695454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.700779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa56197-912b-40d6-adc4-131e18ffbfb5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.702303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.711173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22257e8b-37ac-42d3-be65-ceaf7a83e4ee None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.736743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7d41c-6da3-4675-8fa5-459377ae2f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.737300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d41c-6da3-4675-8fa5-459377ae2f7b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.737583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d41c-6da3-4675-8fa5-459377ae2f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.737854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7d41c-6da3-4675-8fa5-459377ae2f7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.745804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62308ac8-c9c0-4f0f-9972-b6ef8b1b4b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.746413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62308ac8-c9c0-4f0f-9972-b6ef8b1b4b57 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.746696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62308ac8-c9c0-4f0f-9972-b6ef8b1b4b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.746973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62308ac8-c9c0-4f0f-9972-b6ef8b1b4b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.747706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7d41c-6da3-4675-8fa5-459377ae2f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-6ee7d41c-6da3-4675-8fa5-459377ae2f7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:09.752389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:09.753255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:09.754007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 40/81] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:09.754602 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7d41c-6da3-4675-8fa5-459377ae2f7b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.814021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62308ac8-c9c0-4f0f-9972-b6ef8b1b4b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:09.815506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62308ac8-c9c0-4f0f-9972-b6ef8b1b4b57 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:09.853378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7d41c-6da3-4675-8fa5-459377ae2f7b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:09.887633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6b86-c26e-43ee-9067-cc45cc4b0697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:09.888238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6b86-c26e-43ee-9067-cc45cc4b0697 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:09.888550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6b86-c26e-43ee-9067-cc45cc4b0697 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:09.888867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba6b86-c26e-43ee-9067-cc45cc4b0697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:09.898453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.914658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 42/82] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:09.915534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62308ac8-c9c0-4f0f-9972-b6ef8b1b4b57 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:09.923889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:09.964218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba6b86-c26e-43ee-9067-cc45cc4b0697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:09.966806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba6b86-c26e-43ee-9067-cc45cc4b0697 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.011456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62308ac8-c9c0-4f0f-9972-b6ef8b1b4b57 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.045913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ddd56-f0b5-4a71-b4ca-df32815f9215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.046515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ddd56-f0b5-4a71-b4ca-df32815f9215 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.046795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ddd56-f0b5-4a71-b4ca-df32815f9215 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.047069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ddd56-f0b5-4a71-b4ca-df32815f9215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.064159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 41/83] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:10.065030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6b86-c26e-43ee-9067-cc45cc4b0697 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.106487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:10.119431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ddd56-f0b5-4a71-b4ca-df32815f9215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.120957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ddd56-f0b5-4a71-b4ca-df32815f9215 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.124450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:10.163625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba6b86-c26e-43ee-9067-cc45cc4b0697 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.197711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4263be-242e-47a3-acf0-fb29e981f570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.198439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4263be-242e-47a3-acf0-fb29e981f570 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.198722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4263be-242e-47a3-acf0-fb29e981f570 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.198994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4263be-242e-47a3-acf0-fb29e981f570 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.200846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4263be-242e-47a3-acf0-fb29e981f570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.202315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4263be-242e-47a3-acf0-fb29e981f570 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.203909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 42/84] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:10.204698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4263be-242e-47a3-acf0-fb29e981f570 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.221995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 43/85] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:10.222824 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ddd56-f0b5-4a71-b4ca-df32815f9215 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.300199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4263be-242e-47a3-acf0-fb29e981f570 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.303903 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:10.319290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ddd56-f0b5-4a71-b4ca-df32815f9215 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.329328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:10.335632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.336244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.336559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.336835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.337503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:10.337803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.339166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.347481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 43/86] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:10.348260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.354161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ed64d-2e5b-4106-8722-de86d95d3867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.354789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ed64d-2e5b-4106-8722-de86d95d3867 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.355064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ed64d-2e5b-4106-8722-de86d95d3867 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.355336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ed64d-2e5b-4106-8722-de86d95d3867 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.356516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ed64d-2e5b-4106-8722-de86d95d3867 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.357957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ed64d-2e5b-4106-8722-de86d95d3867 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.359591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 44/87] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:10.360319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ed64d-2e5b-4106-8722-de86d95d3867 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.455197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57966-df8f-48e0-a169-aec6ccb223fa None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.459270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ed64d-2e5b-4106-8722-de86d95d3867 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.489602 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478aa16c-f6d3-4825-a3f1-53f086fceac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.490233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478aa16c-f6d3-4825-a3f1-53f086fceac6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.490513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478aa16c-f6d3-4825-a3f1-53f086fceac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.490789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478aa16c-f6d3-4825-a3f1-53f086fceac6 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.491899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478aa16c-f6d3-4825-a3f1-53f086fceac6 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.492939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebb174-d03a-4eb4-bb55-457b0bb5ae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.493372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478aa16c-f6d3-4825-a3f1-53f086fceac6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.493549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb174-d03a-4eb4-bb55-457b0bb5ae34 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.493833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb174-d03a-4eb4-bb55-457b0bb5ae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.494108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebb174-d03a-4eb4-bb55-457b0bb5ae34 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.494752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebb174-d03a-4eb4-bb55-457b0bb5ae34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.494980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 44/88] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:10.495739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478aa16c-f6d3-4825-a3f1-53f086fceac6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.496194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebb174-d03a-4eb4-bb55-457b0bb5ae34 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.519248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 45/89] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:00:10 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:10.520461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebb174-d03a-4eb4-bb55-457b0bb5ae34 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.536052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:10.574187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:10.627164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478aa16c-f6d3-4825-a3f1-53f086fceac6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.650642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebb174-d03a-4eb4-bb55-457b0bb5ae34 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.670073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a2487-32f4-4069-9bf2-c70b022c9172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.671342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a2487-32f4-4069-9bf2-c70b022c9172 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.671846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a2487-32f4-4069-9bf2-c70b022c9172 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.672134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a2487-32f4-4069-9bf2-c70b022c9172 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.673839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a2487-32f4-4069-9bf2-c70b022c9172 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.675665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a2487-32f4-4069-9bf2-c70b022c9172 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.677872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 45/90] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:10.678657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a2487-32f4-4069-9bf2-c70b022c9172 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.690682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2767d-2275-4f4a-85c0-d85c6abfb711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.691316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2767d-2275-4f4a-85c0-d85c6abfb711 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.691593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2767d-2275-4f4a-85c0-d85c6abfb711 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.691868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2767d-2275-4f4a-85c0-d85c6abfb711 None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.693018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2767d-2275-4f4a-85c0-d85c6abfb711 None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.694532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2767d-2275-4f4a-85c0-d85c6abfb711 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.696179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 46/91] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:10.697062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2767d-2275-4f4a-85c0-d85c6abfb711 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.800796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:10.829287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2767d-2275-4f4a-85c0-d85c6abfb711 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.830985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a2487-32f4-4069-9bf2-c70b022c9172 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:10.864264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:10.875260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.875446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:10.875876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.875995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:10.876149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.876272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:10.876507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.876619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:10.877188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=57d897223df84e6ba0e4445299c6559e, user.id=910b0103ce4a49babfe0b9390ae4fdd2) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:10.877347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] RBAC: Adding query filter params (role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=57d897223df84e6ba0e4445299c6559e, user.id=832165b8104e4cafa72e1d439a9295d0) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:10.877494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.877637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:10.878934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.879099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:10.884983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 47/92] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/role_assignments?user.id=910b0103ce4a49babfe0b9390ae4fdd2&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:10.885810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:10.896808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 46/93] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/role_assignments?user.id=832165b8104e4cafa72e1d439a9295d0&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:10.899387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:11.088749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7328f-6c9c-4ad3-8059-1dc8837b2618 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:11.127346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:11.128029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:11.128316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:11.128631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:11.129727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] RBAC: Adding query filter params (role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=57d897223df84e6ba0e4445299c6559e, user.id=832165b8104e4cafa72e1d439a9295d0) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:11.130051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:11.131728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:11.139207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 48/94] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/role_assignments?user.id=832165b8104e4cafa72e1d439a9295d0&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:11.166418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac3dad-0e64-4da8-9421-efb3c4351627 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:11.218037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc671c-9fad-47d5-8249-96fbbb758e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:11.219054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc671c-9fad-47d5-8249-96fbbb758e83 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:11.219541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc671c-9fad-47d5-8249-96fbbb758e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:11.220016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc671c-9fad-47d5-8249-96fbbb758e83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:11.309858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:11.330395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc671c-9fad-47d5-8249-96fbbb758e83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:11.332263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc671c-9fad-47d5-8249-96fbbb758e83 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:11.444211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 47/95] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:11.452471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:11.469040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:11.613535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bd3cf-418f-4387-be47-b735797839c1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:11.666065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda33921-39cf-4c33-9168-13b580ae8dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:11.666813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda33921-39cf-4c33-9168-13b580ae8dad None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:11.666990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda33921-39cf-4c33-9168-13b580ae8dad None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:11.667271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda33921-39cf-4c33-9168-13b580ae8dad None admin] PATH_INFO: `/v3/projects/96d24afaa5cb4cb496972b69a68298ed` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:11.669011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda33921-39cf-4c33-9168-13b580ae8dad None admin] RBAC: Authorizing `identity:get_project(project_id=96d24afaa5cb4cb496972b69a68298ed)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:11.670465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda33921-39cf-4c33-9168-13b580ae8dad None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:11.672421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 49/96] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:11 2022] GET /identity/v3/projects/96d24afaa5cb4cb496972b69a68298ed => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:11.678111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc671c-9fad-47d5-8249-96fbbb758e83 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:11.811673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:11.820592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc671c-9fad-47d5-8249-96fbbb758e83 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:11.870036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c80afc-4071-411f-945b-7f1e937189b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:11.870651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c80afc-4071-411f-945b-7f1e937189b9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:11.870927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c80afc-4071-411f-945b-7f1e937189b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:11.871772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c80afc-4071-411f-945b-7f1e937189b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:11.872600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c80afc-4071-411f-945b-7f1e937189b9 None admin] RBAC: Adding query filter params (role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=96d24afaa5cb4cb496972b69a68298ed, user.id=832165b8104e4cafa72e1d439a9295d0) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:11.872909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c80afc-4071-411f-945b-7f1e937189b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:11.874354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c80afc-4071-411f-945b-7f1e937189b9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:11.884607 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 48/97] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:11 2022] GET /identity/v3/role_assignments?user.id=832165b8104e4cafa72e1d439a9295d0&scope.project.id=96d24afaa5cb4cb496972b69a68298ed&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:13.576979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfae68-57df-4fd5-a061-2aa72c7376a4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:13.577475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfae68-57df-4fd5-a061-2aa72c7376a4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:13.577544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfae68-57df-4fd5-a061-2aa72c7376a4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:13.579152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 50/98] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:13.584217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940a26a-df1f-4c4b-b39f-c87a72abdad2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:13.585019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940a26a-df1f-4c4b-b39f-c87a72abdad2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:13.585482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940a26a-df1f-4c4b-b39f-c87a72abdad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:13.607024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befed61-ac66-48c3-a576-0e8920a5accb None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:13.607345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befed61-ac66-48c3-a576-0e8920a5accb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:13.607592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befed61-ac66-48c3-a576-0e8920a5accb None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:13.608993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 51/99] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:13.611523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9940a26a-df1f-4c4b-b39f-c87a72abdad2 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:13.613026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9940a26a-df1f-4c4b-b39f-c87a72abdad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:13.614117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18bb64-c0f8-4bdf-ab72-b24e4db82693 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:13.614406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18bb64-c0f8-4bdf-ab72-b24e4db82693 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:13.614684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18bb64-c0f8-4bdf-ab72-b24e4db82693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:13.643364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ce18bb64-c0f8-4bdf-ab72-b24e4db82693 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:13.644513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce18bb64-c0f8-4bdf-ab72-b24e4db82693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:13.778853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 49/100] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:13.798310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 52/101] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:13.834450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940a26a-df1f-4c4b-b39f-c87a72abdad2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:13.834629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18bb64-c0f8-4bdf-ab72-b24e4db82693 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:13.837810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.010903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.011099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18bb64-c0f8-4bdf-ab72-b24e4db82693 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.032864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940a26a-df1f-4c4b-b39f-c87a72abdad2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.053379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11643a3b-2bcb-46b4-8ef9-7a56ab0a68af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.054028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11643a3b-2bcb-46b4-8ef9-7a56ab0a68af None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.054367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11643a3b-2bcb-46b4-8ef9-7a56ab0a68af None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.054681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11643a3b-2bcb-46b4-8ef9-7a56ab0a68af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.069069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11643a3b-2bcb-46b4-8ef9-7a56ab0a68af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:14.071529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a005f94-a7bb-4613-a1bb-5fc7e8e6ea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.072173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005f94-a7bb-4613-a1bb-5fc7e8e6ea42 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.072469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005f94-a7bb-4613-a1bb-5fc7e8e6ea42 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.072757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005f94-a7bb-4613-a1bb-5fc7e8e6ea42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-11643a3b-2bcb-46b4-8ef9-7a56ab0a68af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:14.074103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:14.075759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:14.076593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 53/102] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:14.080332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc6604-9794-4502-ba2a-1db42d08b0ae None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.080332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc6604-9794-4502-ba2a-1db42d08b0ae None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.080332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc6604-9794-4502-ba2a-1db42d08b0ae None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.080332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 54/103] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:14.081366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636453-85db-4df4-bbd8-45eb01152a66 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.081650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636453-85db-4df4-bbd8-45eb01152a66 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.081924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636453-85db-4df4-bbd8-45eb01152a66 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.083427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 55/104] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:14.088530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496f368-0a30-40cf-a9a9-f499080c5ee9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.088817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496f368-0a30-40cf-a9a9-f499080c5ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.089189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496f368-0a30-40cf-a9a9-f499080c5ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.090084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a005f94-a7bb-4613-a1bb-5fc7e8e6ea42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-6a005f94-a7bb-4613-a1bb-5fc7e8e6ea42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:14.095123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:14.097139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:14.098981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 50/105] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:14.104503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f0d83-aec6-471a-8a3d-afffa56149b3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.104503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f0d83-aec6-471a-8a3d-afffa56149b3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.104503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f0d83-aec6-471a-8a3d-afffa56149b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.109017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8496f368-0a30-40cf-a9a9-f499080c5ee9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:14.110317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8496f368-0a30-40cf-a9a9-f499080c5ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:14.121046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-836f0d83-aec6-471a-8a3d-afffa56149b3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:14.122769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-836f0d83-aec6-471a-8a3d-afffa56149b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:14.177392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.190433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 56/106] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:14.191271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496f368-0a30-40cf-a9a9-f499080c5ee9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.191927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 51/107] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:14.192854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f0d83-aec6-471a-8a3d-afffa56149b3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.310115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.334787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496f368-0a30-40cf-a9a9-f499080c5ee9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.338472 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f0d83-aec6-471a-8a3d-afffa56149b3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.369936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.370557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.370835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.371950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.372700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:14.373094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:14.374448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:14.385106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 57/108] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:14.385106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.389135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.389859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.390141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.390408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.391083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:14.391385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:14.392799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:14.406402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 52/109] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:14.406402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.457505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.506630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0bd81-4dca-4451-b800-2c5a58b6fad6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.510691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad01bce-75e9-4b9d-97ad-914941964ef2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.541691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3032ad-c83c-4923-8494-7e9498ba143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.542553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3032ad-c83c-4923-8494-7e9498ba143e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.543510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3032ad-c83c-4923-8494-7e9498ba143e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.543786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3032ad-c83c-4923-8494-7e9498ba143e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.546639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5add72-6133-45bb-a320-79ef346bee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.547274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5add72-6133-45bb-a320-79ef346bee75 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.547557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5add72-6133-45bb-a320-79ef346bee75 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.547830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5add72-6133-45bb-a320-79ef346bee75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.549035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5add72-6133-45bb-a320-79ef346bee75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:14.550481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5add72-6133-45bb-a320-79ef346bee75 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:14.552247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 53/110] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:14.553268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fba0c3-8a3f-4b3e-851c-86b58c7567ba None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.553631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fba0c3-8a3f-4b3e-851c-86b58c7567ba None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.553912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fba0c3-8a3f-4b3e-851c-86b58c7567ba None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.555259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 54/111] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:14.555831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fba0c3-8a3f-4b3e-851c-86b58c7567ba None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.563336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3032ad-c83c-4923-8494-7e9498ba143e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:14.567472 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7e3032ad-c83c-4923-8494-7e9498ba143e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:14.569849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:14.570797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:14.571740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:14.573701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 58/112] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:14.578795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3032ad-c83c-4923-8494-7e9498ba143e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.668710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fba0c3-8a3f-4b3e-851c-86b58c7567ba None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.693297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3032ad-c83c-4923-8494-7e9498ba143e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.704492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e05364-6d9a-43bd-9875-0d2413fc4c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.705313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05364-6d9a-43bd-9875-0d2413fc4c4d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.705733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05364-6d9a-43bd-9875-0d2413fc4c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.706144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e05364-6d9a-43bd-9875-0d2413fc4c4d None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.707418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e05364-6d9a-43bd-9875-0d2413fc4c4d None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:14.709023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e05364-6d9a-43bd-9875-0d2413fc4c4d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:14.710724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 55/113] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:14.710875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.712137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc87bb7-4283-40cc-b473-9b887ed8a117 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.712745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc87bb7-4283-40cc-b473-9b887ed8a117 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.713286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc87bb7-4283-40cc-b473-9b887ed8a117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.732055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2dc87bb7-4283-40cc-b473-9b887ed8a117 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:14.733579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2dc87bb7-4283-40cc-b473-9b887ed8a117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:14.734851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f72a1-db94-4508-b734-b49d6e133e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.735487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f72a1-db94-4508-b734-b49d6e133e22 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.735775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f72a1-db94-4508-b734-b49d6e133e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.736041 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f72a1-db94-4508-b734-b49d6e133e22 None admin] PATH_INFO: `/v3/users/74fa483d9f204772b821eba2b9690718` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.750386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f72a1-db94-4508-b734-b49d6e133e22 None admin] RBAC: Authorizing `identity:get_user(user_id=74fa483d9f204772b821eba2b9690718)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:14.751842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f72a1-db94-4508-b734-b49d6e133e22 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:14.757957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 59/114] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/users/74fa483d9f204772b821eba2b9690718 => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:14.757957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f72a1-db94-4508-b734-b49d6e133e22 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.813319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 56/115] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:14.814590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc87bb7-4283-40cc-b473-9b887ed8a117 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.821775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.878060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f72a1-db94-4508-b734-b49d6e133e22 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.919170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75877a00-917c-4c95-b780-ea3effbf63cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.919778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75877a00-917c-4c95-b780-ea3effbf63cc None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.920057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75877a00-917c-4c95-b780-ea3effbf63cc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.920327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75877a00-917c-4c95-b780-ea3effbf63cc None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.921005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75877a00-917c-4c95-b780-ea3effbf63cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:14.922475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75877a00-917c-4c95-b780-ea3effbf63cc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:14.934413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc87bb7-4283-40cc-b473-9b887ed8a117 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:14.948118 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 60/116] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:00:14 2022] POST /identity/v3/projects => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:14.949229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75877a00-917c-4c95-b780-ea3effbf63cc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:14.973797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:14.974440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:14.974723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:14.974998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:14.975692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:14.975990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:14.977451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:14.986668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:14.992756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 57/117] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:14.992756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.067046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:15.094136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75877a00-917c-4c95-b780-ea3effbf63cc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:15.110334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429c8af-5d07-42b6-b016-9aa7032f455f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:15.134766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbd0c-f5eb-415d-bcc2-593ebd10448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:15.135562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbd0c-f5eb-415d-bcc2-593ebd10448f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.135938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbd0c-f5eb-415d-bcc2-593ebd10448f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.136335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfbd0c-f5eb-415d-bcc2-593ebd10448f None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.138336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfbd0c-f5eb-415d-bcc2-593ebd10448f None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:15.139928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfbd0c-f5eb-415d-bcc2-593ebd10448f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:15.141873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 61/118] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:15.142920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbd0c-f5eb-415d-bcc2-593ebd10448f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.153517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3719da3-2087-40ad-8b9a-ee14dd40b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:15.154535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719da3-2087-40ad-8b9a-ee14dd40b749 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.154818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719da3-2087-40ad-8b9a-ee14dd40b749 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.155097 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719da3-2087-40ad-8b9a-ee14dd40b749 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.228292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3719da3-2087-40ad-8b9a-ee14dd40b749 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:15.229789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3719da3-2087-40ad-8b9a-ee14dd40b749 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:15.238916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:15.247643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfbd0c-f5eb-415d-bcc2-593ebd10448f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:15.286345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cac219-9a6b-479f-bcc3-79d1583b2c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:15.287106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cac219-9a6b-479f-bcc3-79d1583b2c59 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.287396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cac219-9a6b-479f-bcc3-79d1583b2c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.287721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cac219-9a6b-479f-bcc3-79d1583b2c59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.299622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cac219-9a6b-479f-bcc3-79d1583b2c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a5cac219-9a6b-479f-bcc3-79d1583b2c59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:15.304959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:15.305981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:15.306931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:15.308070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:15.312224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 62/119] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:15.313585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c79c-ea6c-458f-a1e0-dc176db31307 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.313815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c79c-ea6c-458f-a1e0-dc176db31307 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.314088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c79c-ea6c-458f-a1e0-dc176db31307 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.315392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 63/120] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:15.316139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814c79c-ea6c-458f-a1e0-dc176db31307 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.353058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 58/121] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:15.353801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3719da3-2087-40ad-8b9a-ee14dd40b749 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.441704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814c79c-ea6c-458f-a1e0-dc176db31307 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:15.472984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3719da3-2087-40ad-8b9a-ee14dd40b749 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:15.477512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989d1fa-068c-437a-b6b2-e27a98ef8064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:15.478152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989d1fa-068c-437a-b6b2-e27a98ef8064 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.478433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989d1fa-068c-437a-b6b2-e27a98ef8064 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.478710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989d1fa-068c-437a-b6b2-e27a98ef8064 None admin] PATH_INFO: `/v3/users/910b0103ce4a49babfe0b9390ae4fdd2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.479893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989d1fa-068c-437a-b6b2-e27a98ef8064 None admin] RBAC: Authorizing `identity:get_user(user_id=910b0103ce4a49babfe0b9390ae4fdd2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:15.482342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989d1fa-068c-437a-b6b2-e27a98ef8064 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:15.488214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 64/122] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:15 2022] GET /identity/v3/users/910b0103ce4a49babfe0b9390ae4fdd2 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:15.488214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989d1fa-068c-437a-b6b2-e27a98ef8064 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.516648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:15.518924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7608db-8dee-4c4c-ab6d-173056a926c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:15.519678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7608db-8dee-4c4c-ab6d-173056a926c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.520028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7608db-8dee-4c4c-ab6d-173056a926c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.520312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7608db-8dee-4c4c-ab6d-173056a926c6 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e/users/832165b8104e4cafa72e1d439a9295d0/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.522444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7608db-8dee-4c4c-ab6d-173056a926c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=57d897223df84e6ba0e4445299c6559e, user_id=832165b8104e4cafa72e1d439a9295d0, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:15.523947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7608db-8dee-4c4c-ab6d-173056a926c6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:15.541124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 59/123] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:15 2022] PUT /identity/v3/projects/57d897223df84e6ba0e4445299c6559e/users/832165b8104e4cafa72e1d439a9295d0/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:15.544592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4476adc-9626-48fc-8a17-694873a9ec57 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.544592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4476adc-9626-48fc-8a17-694873a9ec57 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.544592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4476adc-9626-48fc-8a17-694873a9ec57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.565438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a4476adc-9626-48fc-8a17-694873a9ec57 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:15.566991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a4476adc-9626-48fc-8a17-694873a9ec57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:15.593650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:15.640010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989d1fa-068c-437a-b6b2-e27a98ef8064 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:15.656185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 60/124] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:15.657487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4476adc-9626-48fc-8a17-694873a9ec57 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.692738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:15.693364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.693642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.693920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.694595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:15.694898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:15.696307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:15.705191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 65/125] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:15.706067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.809900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:15.810965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4476adc-9626-48fc-8a17-694873a9ec57 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:15.830390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a7a9-fad7-47a8-907e-c50f019273c9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:15.864280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4e926-78f1-42ea-8516-8a0b4819a993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:15.865243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4e926-78f1-42ea-8516-8a0b4819a993 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.865529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4e926-78f1-42ea-8516-8a0b4819a993 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.865805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4e926-78f1-42ea-8516-8a0b4819a993 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.867605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4e926-78f1-42ea-8516-8a0b4819a993 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:15.869383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4e926-78f1-42ea-8516-8a0b4819a993 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:15.871863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 61/126] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:15 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:15.872450 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87baea-8ded-4c58-8fed-53d677fe390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:15.873132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4e926-78f1-42ea-8516-8a0b4819a993 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.873255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87baea-8ded-4c58-8fed-53d677fe390b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:15.873431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87baea-8ded-4c58-8fed-53d677fe390b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:15.873708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87baea-8ded-4c58-8fed-53d677fe390b None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:15.874848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87baea-8ded-4c58-8fed-53d677fe390b None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:15.876290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87baea-8ded-4c58-8fed-53d677fe390b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:15.878270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 66/127] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:15 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:15.879097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87baea-8ded-4c58-8fed-53d677fe390b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:15.906282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:16.000014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4e926-78f1-42ea-8516-8a0b4819a993 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:16.000374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87baea-8ded-4c58-8fed-53d677fe390b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:16.043448 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daa1c5-d439-4c67-aedc-cf94e9838c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:16.044372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daa1c5-d439-4c67-aedc-cf94e9838c1b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:16.044745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daa1c5-d439-4c67-aedc-cf94e9838c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:16.045032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daa1c5-d439-4c67-aedc-cf94e9838c1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:16.047968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77534d14-8350-4d59-8607-f90cb333511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:16.048895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534d14-8350-4d59-8607-f90cb333511a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:16.049371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534d14-8350-4d59-8607-f90cb333511a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:16.049871 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534d14-8350-4d59-8607-f90cb333511a None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:16.051805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77534d14-8350-4d59-8607-f90cb333511a None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:16.054300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77534d14-8350-4d59-8607-f90cb333511a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:16.056532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 67/128] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:15 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:16.057628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77534d14-8350-4d59-8607-f90cb333511a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:16.062303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58daa1c5-d439-4c67-aedc-cf94e9838c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-58daa1c5-d439-4c67-aedc-cf94e9838c1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:16.071611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:16.072554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 62/129] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:16.073316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daa1c5-d439-4c67-aedc-cf94e9838c1b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:16.096153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:16.177806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:16.188753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77534d14-8350-4d59-8607-f90cb333511a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:16.197627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daa1c5-d439-4c67-aedc-cf94e9838c1b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:16.224417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:16.225066 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:16.225335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:16.225675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:16.226389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] RBAC: Adding query filter params (role.id=585290ba3e014e9e9ae58a7c1d1f1f67, scope.project.id=57d897223df84e6ba0e4445299c6559e, user.id=74fa483d9f204772b821eba2b9690718) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:16.226700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:16.228258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:16.235148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 68/130] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:16 2022] GET /identity/v3/role_assignments?user.id=74fa483d9f204772b821eba2b9690718&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=585290ba3e014e9e9ae58a7c1d1f1f67 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:16.236222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:16.237343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ed401-3d0a-477e-afc9-5372948c15a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:16.238220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ed401-3d0a-477e-afc9-5372948c15a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:16.238670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ed401-3d0a-477e-afc9-5372948c15a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:16.239088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ed401-3d0a-477e-afc9-5372948c15a9 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e/users/910b0103ce4a49babfe0b9390ae4fdd2/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:16.248498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ed401-3d0a-477e-afc9-5372948c15a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=57d897223df84e6ba0e4445299c6559e, user_id=910b0103ce4a49babfe0b9390ae4fdd2, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:16.248498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ed401-3d0a-477e-afc9-5372948c15a9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:16.275135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 63/131] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:16 2022] PUT /identity/v3/projects/57d897223df84e6ba0e4445299c6559e/users/910b0103ce4a49babfe0b9390ae4fdd2/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:16.275763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ed401-3d0a-477e-afc9-5372948c15a9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:16.419376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:16.450772 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee779f-f35f-4aa9-ab41-9d2651b2fd09 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:16.507102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ed401-3d0a-477e-afc9-5372948c15a9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:16.509514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17597ec-c3b7-4027-b2bb-fef4818c828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:16.510246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17597ec-c3b7-4027-b2bb-fef4818c828d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:16.510529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17597ec-c3b7-4027-b2bb-fef4818c828d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:16.510802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17597ec-c3b7-4027-b2bb-fef4818c828d None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:16.514641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17597ec-c3b7-4027-b2bb-fef4818c828d None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:16.516121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17597ec-c3b7-4027-b2bb-fef4818c828d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:16.517833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 69/132] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:16 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:16.523523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17597ec-c3b7-4027-b2bb-fef4818c828d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:16.584843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:16.585465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:16.585745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:16.586025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:16.586763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:16.587069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:16.588528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:16.604622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 64/133] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:16.614540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:16.637391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:16.677464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17597ec-c3b7-4027-b2bb-fef4818c828d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:16.721260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a515658-a797-4c0a-94bf-81011ce6e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:16.721869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515658-a797-4c0a-94bf-81011ce6e66b None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:16.722154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515658-a797-4c0a-94bf-81011ce6e66b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:16.722437 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515658-a797-4c0a-94bf-81011ce6e66b None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e/users/832165b8104e4cafa72e1d439a9295d0/roles/199fa373d2ff46b7a0c430b4a8c056e9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:16.729787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a515658-a797-4c0a-94bf-81011ce6e66b None admin] RBAC: Authorizing `identity:create_grant(project_id=57d897223df84e6ba0e4445299c6559e, user_id=832165b8104e4cafa72e1d439a9295d0, role_id=199fa373d2ff46b7a0c430b4a8c056e9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:16.731720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a515658-a797-4c0a-94bf-81011ce6e66b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:16.758337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa9a81-c6bd-4035-a2bf-fc3913cec041 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:16.764040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 70/134] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:16 2022] PUT /identity/v3/projects/57d897223df84e6ba0e4445299c6559e/users/832165b8104e4cafa72e1d439a9295d0/roles/199fa373d2ff46b7a0c430b4a8c056e9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:00:16.802877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04208d-a80e-47d6-8734-02cdaedef4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:16.803914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04208d-a80e-47d6-8734-02cdaedef4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:16.804388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04208d-a80e-47d6-8734-02cdaedef4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:16.804910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04208d-a80e-47d6-8734-02cdaedef4e4 None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:16.806235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04208d-a80e-47d6-8734-02cdaedef4e4 None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:16.808035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04208d-a80e-47d6-8734-02cdaedef4e4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:16.811159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 65/135] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:16 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:16.817419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a515658-a797-4c0a-94bf-81011ce6e66b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:16.835835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:16.957780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a515658-a797-4c0a-94bf-81011ce6e66b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:17.008430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dcb4b-a1be-4fa4-ab83-2bd2f638892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:17.009441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcb4b-a1be-4fa4-ab83-2bd2f638892b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:17.009913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcb4b-a1be-4fa4-ab83-2bd2f638892b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:17.010378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dcb4b-a1be-4fa4-ab83-2bd2f638892b None admin] PATH_INFO: `/v3/projects/96d24afaa5cb4cb496972b69a68298ed` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:17.012667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dcb4b-a1be-4fa4-ab83-2bd2f638892b None admin] RBAC: Authorizing `identity:get_project(project_id=96d24afaa5cb4cb496972b69a68298ed)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:17.014458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dcb4b-a1be-4fa4-ab83-2bd2f638892b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:17.016387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 71/136] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:16 2022] GET /identity/v3/projects/96d24afaa5cb4cb496972b69a68298ed => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:17.023747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04208d-a80e-47d6-8734-02cdaedef4e4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:17.125063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:17.154589 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04208d-a80e-47d6-8734-02cdaedef4e4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:17.204145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0563f-e24c-4e70-b5fe-5d56cbac6c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:17.204848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0563f-e24c-4e70-b5fe-5d56cbac6c21 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:17.205123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0563f-e24c-4e70-b5fe-5d56cbac6c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:17.205396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0563f-e24c-4e70-b5fe-5d56cbac6c21 None admin] PATH_INFO: `/v3/projects/96d24afaa5cb4cb496972b69a68298ed/users/832165b8104e4cafa72e1d439a9295d0/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:17.209737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0563f-e24c-4e70-b5fe-5d56cbac6c21 None admin] RBAC: Authorizing `identity:create_grant(project_id=96d24afaa5cb4cb496972b69a68298ed, user_id=832165b8104e4cafa72e1d439a9295d0, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:17.211206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0563f-e24c-4e70-b5fe-5d56cbac6c21 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:17.236792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 66/137] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:17 2022] PUT /identity/v3/projects/96d24afaa5cb4cb496972b69a68298ed/users/832165b8104e4cafa72e1d439a9295d0/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:17.324581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b6db-af6a-49aa-aa83-6262b9c87dab None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:17.324807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b6db-af6a-49aa-aa83-6262b9c87dab None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:17.325147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b6db-af6a-49aa-aa83-6262b9c87dab None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:17.326990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 72/138] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:17.333750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41821849-f3c4-4668-8fbe-da3ec2d74987 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:17.334348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41821849-f3c4-4668-8fbe-da3ec2d74987 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:17.334825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41821849-f3c4-4668-8fbe-da3ec2d74987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:17.354973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-41821849-f3c4-4668-8fbe-da3ec2d74987 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:17.357564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-41821849-f3c4-4668-8fbe-da3ec2d74987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:17.457612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 67/139] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:17.499380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b6db-af6a-49aa-aa83-6262b9c87dab None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:17.601205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:17.626490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b6db-af6a-49aa-aa83-6262b9c87dab None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:17.670230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598dc3e-f699-44ab-8aa8-4734c9176907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:17.670866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598dc3e-f699-44ab-8aa8-4734c9176907 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:17.671144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598dc3e-f699-44ab-8aa8-4734c9176907 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:17.671413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598dc3e-f699-44ab-8aa8-4734c9176907 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:17.672544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598dc3e-f699-44ab-8aa8-4734c9176907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:17.674012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598dc3e-f699-44ab-8aa8-4734c9176907 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:17.675910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 73/140] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:17.681150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41821849-f3c4-4668-8fbe-da3ec2d74987 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:17.759959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:17.805502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41821849-f3c4-4668-8fbe-da3ec2d74987 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:17.858464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaba8b-03d8-41f9-b33c-8fc2e6ef034e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:17.859065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaba8b-03d8-41f9-b33c-8fc2e6ef034e None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:17.859326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaba8b-03d8-41f9-b33c-8fc2e6ef034e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:17.859600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaba8b-03d8-41f9-b33c-8fc2e6ef034e None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:17.860263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaba8b-03d8-41f9-b33c-8fc2e6ef034e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:17.861726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaba8b-03d8-41f9-b33c-8fc2e6ef034e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:17.899027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 68/141] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:00:17 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:18.681643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa391-0958-4b5f-9f2a-0fa73a31775e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:18.682183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa391-0958-4b5f-9f2a-0fa73a31775e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:18.682183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa391-0958-4b5f-9f2a-0fa73a31775e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:18.683560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 74/142] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:18.693813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b34900-1235-4d76-adf8-4a47a4ed9a3f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:18.694092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b34900-1235-4d76-adf8-4a47a4ed9a3f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:18.694373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b34900-1235-4d76-adf8-4a47a4ed9a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:18.729436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b2b34900-1235-4d76-adf8-4a47a4ed9a3f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:18.731285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b2b34900-1235-4d76-adf8-4a47a4ed9a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:18.828013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 69/143] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:18.854068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fa391-0958-4b5f-9f2a-0fa73a31775e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:18.971380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fa391-0958-4b5f-9f2a-0fa73a31775e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:19.011299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10401b-c854-4422-b9ff-eac375c2463a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:19.012599 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10401b-c854-4422-b9ff-eac375c2463a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:19.012923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10401b-c854-4422-b9ff-eac375c2463a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:19.013198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10401b-c854-4422-b9ff-eac375c2463a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:19.025682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10401b-c854-4422-b9ff-eac375c2463a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2d10401b-c854-4422-b9ff-eac375c2463a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:19.030711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:19.031589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:19.032820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 75/144] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:19.038078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b34900-1235-4d76-adf8-4a47a4ed9a3f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:19.126313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:19.162050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656bcce-eefa-4e63-ae8f-522beebb1d3d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:19.162362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656bcce-eefa-4e63-ae8f-522beebb1d3d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:19.162594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656bcce-eefa-4e63-ae8f-522beebb1d3d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:19.163927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 76/145] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:19.170618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6ddc2-4d8e-45ea-b463-e2132f62be04 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:19.170903 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6ddc2-4d8e-45ea-b463-e2132f62be04 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:19.171146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6ddc2-4d8e-45ea-b463-e2132f62be04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:19.201736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bac6ddc2-4d8e-45ea-b463-e2132f62be04 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:19.202950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bac6ddc2-4d8e-45ea-b463-e2132f62be04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:19.209439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:19.233020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b34900-1235-4d76-adf8-4a47a4ed9a3f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:19.293899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:19.294812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:19.295102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:19.295380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:19.296079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:19.296391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:19.297930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:19.309763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 70/146] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:19.312556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:19.325715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 77/147] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:19.364818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6ddc2-4d8e-45ea-b463-e2132f62be04 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:19.430723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5dfd8-a956-4a1f-adc1-091cf3bb7fc7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:19.456907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:19.476360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678b40c-0cd6-4c6b-aaca-b84b6b70d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:19.477130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b40c-0cd6-4c6b-aaca-b84b6b70d428 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:19.477944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b40c-0cd6-4c6b-aaca-b84b6b70d428 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:19.478233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b40c-0cd6-4c6b-aaca-b84b6b70d428 None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:19.479558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678b40c-0cd6-4c6b-aaca-b84b6b70d428 None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:19.482095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678b40c-0cd6-4c6b-aaca-b84b6b70d428 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:19.484774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 71/148] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:19 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:19.489320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678b40c-0cd6-4c6b-aaca-b84b6b70d428 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:19.589212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6ddc2-4d8e-45ea-b463-e2132f62be04 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:19.618483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678b40c-0cd6-4c6b-aaca-b84b6b70d428 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:19.651365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f18aef-68cd-48fc-9b63-e5aa3f240c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:19.652009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f18aef-68cd-48fc-9b63-e5aa3f240c48 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:19.652319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f18aef-68cd-48fc-9b63-e5aa3f240c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:19.652617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f18aef-68cd-48fc-9b63-e5aa3f240c48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:19.660076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d506df-fba6-41ed-b30c-d4dc8eb83635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:19.660736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d506df-fba6-41ed-b30c-d4dc8eb83635 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:19.661020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d506df-fba6-41ed-b30c-d4dc8eb83635 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:19.661300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d506df-fba6-41ed-b30c-d4dc8eb83635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:19.669977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:19.677852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f18aef-68cd-48fc-9b63-e5aa3f240c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-22f18aef-68cd-48fc-9b63-e5aa3f240c48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:19.684809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:19.687888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:19.689857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 78/149] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:19.697338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f18aef-68cd-48fc-9b63-e5aa3f240c48 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:19.728818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:19.749305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d506df-fba6-41ed-b30c-d4dc8eb83635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:19.750871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d506df-fba6-41ed-b30c-d4dc8eb83635 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:19.902802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 72/150] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:19.904759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb213fd2-21cd-4ed4-9643-f6b47d73450a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:19.905276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb213fd2-21cd-4ed4-9643-f6b47d73450a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:19.905756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb213fd2-21cd-4ed4-9643-f6b47d73450a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:19.907251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 73/151] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:19.912183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb213fd2-21cd-4ed4-9643-f6b47d73450a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:19.955629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f18aef-68cd-48fc-9b63-e5aa3f240c48 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:20.007451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:20.010746 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f6622-3a90-4d82-aabf-833319c3d729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:20.011429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f6622-3a90-4d82-aabf-833319c3d729 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:20.011866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f6622-3a90-4d82-aabf-833319c3d729 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:20.012278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f6622-3a90-4d82-aabf-833319c3d729 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:20.013306 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f6622-3a90-4d82-aabf-833319c3d729 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:20.013619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f6622-3a90-4d82-aabf-833319c3d729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:20.015021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f6622-3a90-4d82-aabf-833319c3d729 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:20.032464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 79/152] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:20.036372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03256166-4ef3-472b-be66-e4a22f9c5e18 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:20.039001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03256166-4ef3-472b-be66-e4a22f9c5e18 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:20.039486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03256166-4ef3-472b-be66-e4a22f9c5e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:20.044863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb213fd2-21cd-4ed4-9643-f6b47d73450a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:20.081924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-03256166-4ef3-472b-be66-e4a22f9c5e18 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:20.083696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-03256166-4ef3-472b-be66-e4a22f9c5e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:20.148042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c6931-7a45-42fc-a3d0-baed9b065e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:20.149050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c6931-7a45-42fc-a3d0-baed9b065e19 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:20.149484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c6931-7a45-42fc-a3d0-baed9b065e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:20.149909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c6931-7a45-42fc-a3d0-baed9b065e19 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:20.160466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c6931-7a45-42fc-a3d0-baed9b065e19 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:20.160466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c6931-7a45-42fc-a3d0-baed9b065e19 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:20.163030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 74/153] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:19 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:20.164324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c6931-7a45-42fc-a3d0-baed9b065e19 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:20.188221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:20.210204 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 80/154] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:20.211107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03256166-4ef3-472b-be66-e4a22f9c5e18 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:20.407210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03256166-4ef3-472b-be66-e4a22f9c5e18 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:20.463312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c6931-7a45-42fc-a3d0-baed9b065e19 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:20.479667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:20.480868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:20.481159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:20.481429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:20.482213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] RBAC: Adding query filter params (role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=57d897223df84e6ba0e4445299c6559e, user.id=832165b8104e4cafa72e1d439a9295d0) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:20.482520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:20.483994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:20.500071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 81/155] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:20 2022] GET /identity/v3/role_assignments?user.id=832165b8104e4cafa72e1d439a9295d0&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:20.501152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:20.532822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2ddfc-bf4b-4f7d-9459-28b5e2c5befb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:20.533895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2ddfc-bf4b-4f7d-9459-28b5e2c5befb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:20.534393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2ddfc-bf4b-4f7d-9459-28b5e2c5befb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:20.534869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2ddfc-bf4b-4f7d-9459-28b5e2c5befb None admin] PATH_INFO: `/v3/users/74fa483d9f204772b821eba2b9690718` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:20.541562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2ddfc-bf4b-4f7d-9459-28b5e2c5befb None admin] RBAC: Authorizing `identity:get_user(user_id=74fa483d9f204772b821eba2b9690718)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:20.544494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2ddfc-bf4b-4f7d-9459-28b5e2c5befb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:20.546574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 75/156] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:20 2022] GET /identity/v3/users/74fa483d9f204772b821eba2b9690718 => generated 289 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:20.553169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2ddfc-bf4b-4f7d-9459-28b5e2c5befb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:20.590046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:20.632409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:20.718850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a9d89-7517-4dd5-b0e3-666c3e1ac830 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:20.744089 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2ddfc-bf4b-4f7d-9459-28b5e2c5befb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:20.789625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992185c-84e0-4558-865e-6be5f93f6e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:20.790269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992185c-84e0-4558-865e-6be5f93f6e3a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:20.790545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992185c-84e0-4558-865e-6be5f93f6e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:20.790809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992185c-84e0-4558-865e-6be5f93f6e3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:20.817596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992185c-84e0-4558-865e-6be5f93f6e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c992185c-84e0-4558-865e-6be5f93f6e3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:20.822921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:20.824011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:20.825912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 82/157] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:20.833560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992185c-84e0-4558-865e-6be5f93f6e3a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:20.842087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2bfd6-2b2f-46e0-a57b-0dc400f8e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:20.842752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2bfd6-2b2f-46e0-a57b-0dc400f8e8be None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:20.843034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2bfd6-2b2f-46e0-a57b-0dc400f8e8be None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:20.843314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2bfd6-2b2f-46e0-a57b-0dc400f8e8be None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:20.849364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2bfd6-2b2f-46e0-a57b-0dc400f8e8be None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:20.850864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2bfd6-2b2f-46e0-a57b-0dc400f8e8be None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:20.852843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 76/158] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:20 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:20.856957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc28d53-1386-4462-a65c-5fafa5d5dc9e None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:20.857609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc28d53-1386-4462-a65c-5fafa5d5dc9e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:20.857804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc28d53-1386-4462-a65c-5fafa5d5dc9e None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:20.859136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 77/159] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:20.866098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc28d53-1386-4462-a65c-5fafa5d5dc9e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:21.017608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:21.018126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992185c-84e0-4558-865e-6be5f93f6e3a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:21.033743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:21.039919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc28d53-1386-4462-a65c-5fafa5d5dc9e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:21.073303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1042b-b747-4310-b1ec-1ec28441705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:21.073944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1042b-b747-4310-b1ec-1ec28441705b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.074819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1042b-b747-4310-b1ec-1ec28441705b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.075101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1042b-b747-4310-b1ec-1ec28441705b None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.075794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1042b-b747-4310-b1ec-1ec28441705b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:21.076159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1042b-b747-4310-b1ec-1ec28441705b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:21.077634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1042b-b747-4310-b1ec-1ec28441705b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:21.083791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beb52b-0808-44e9-a9f0-a8199cfacfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:21.085107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb52b-0808-44e9-a9f0-a8199cfacfcd None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.085695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb52b-0808-44e9-a9f0-a8199cfacfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.086171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb52b-0808-44e9-a9f0-a8199cfacfcd None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e/users/74fa483d9f204772b821eba2b9690718/roles/585290ba3e014e9e9ae58a7c1d1f1f67` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.086547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 83/160] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:21.088644 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beb52b-0808-44e9-a9f0-a8199cfacfcd None admin] RBAC: Authorizing `identity:create_grant(project_id=57d897223df84e6ba0e4445299c6559e, user_id=74fa483d9f204772b821eba2b9690718, role_id=585290ba3e014e9e9ae58a7c1d1f1f67)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:21.088850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907507-5724-4403-8518-7cd50998c1a8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.089138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907507-5724-4403-8518-7cd50998c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.089471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01907507-5724-4403-8518-7cd50998c1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.092028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beb52b-0808-44e9-a9f0-a8199cfacfcd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:21.109925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 78/161] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:20 2022] PUT /identity/v3/projects/57d897223df84e6ba0e4445299c6559e/users/74fa483d9f204772b821eba2b9690718/roles/585290ba3e014e9e9ae58a7c1d1f1f67 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:00:21.111651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49c64-84be-407a-9deb-4a23726f4f45 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.112151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49c64-84be-407a-9deb-4a23726f4f45 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.112672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49c64-84be-407a-9deb-4a23726f4f45 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.114498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 79/162] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:21.115534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b49c64-84be-407a-9deb-4a23726f4f45 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:21.125567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-01907507-5724-4403-8518-7cd50998c1a8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:21.127649 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-01907507-5724-4403-8518-7cd50998c1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:21.259803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b49c64-84be-407a-9deb-4a23726f4f45 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:21.262567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 84/163] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:21.264304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1753-bf3a-4c46-9fc0-79c13c28304c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.265413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1753-bf3a-4c46-9fc0-79c13c28304c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.267990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1753-bf3a-4c46-9fc0-79c13c28304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.305634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8cad1753-bf3a-4c46-9fc0-79c13c28304c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:21.307017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8cad1753-bf3a-4c46-9fc0-79c13c28304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:21.320869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d0a7d-5c34-4b94-8896-c812e6913ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:21.322027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d0a7d-5c34-4b94-8896-c812e6913ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.322525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d0a7d-5c34-4b94-8896-c812e6913ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.323096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d0a7d-5c34-4b94-8896-c812e6913ef7 None admin] PATH_INFO: `/v3/users/910b0103ce4a49babfe0b9390ae4fdd2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.324847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d0a7d-5c34-4b94-8896-c812e6913ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=910b0103ce4a49babfe0b9390ae4fdd2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:21.326933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d0a7d-5c34-4b94-8896-c812e6913ef7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:21.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 80/164] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:21 2022] GET /identity/v3/users/910b0103ce4a49babfe0b9390ae4fdd2 => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:21.330213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d0a7d-5c34-4b94-8896-c812e6913ef7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:21.356846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:21.452794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d0a7d-5c34-4b94-8896-c812e6913ef7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:21.473776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 85/165] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:21.475053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad1753-bf3a-4c46-9fc0-79c13c28304c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:21.493483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff2a2c-a217-4cd8-a7b4-9da0d5a0ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:21.494713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff2a2c-a217-4cd8-a7b4-9da0d5a0ef3a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.495207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff2a2c-a217-4cd8-a7b4-9da0d5a0ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.495684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff2a2c-a217-4cd8-a7b4-9da0d5a0ef3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.508762 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff2a2c-a217-4cd8-a7b4-9da0d5a0ef3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-45ff2a2c-a217-4cd8-a7b4-9da0d5a0ef3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:21.516339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:21.517263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:21.518087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:21.520778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 81/166] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:21.521795 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff2a2c-a217-4cd8-a7b4-9da0d5a0ef3a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:21.531758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:21.615439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:21.664934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff2a2c-a217-4cd8-a7b4-9da0d5a0ef3a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:21.666288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad1753-bf3a-4c46-9fc0-79c13c28304c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:21.709760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b359692-5a25-4d93-aea7-54b981742f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:21.710842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359692-5a25-4d93-aea7-54b981742f05 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.711341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359692-5a25-4d93-aea7-54b981742f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.711894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b359692-5a25-4d93-aea7-54b981742f05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.734806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b359692-5a25-4d93-aea7-54b981742f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4b359692-5a25-4d93-aea7-54b981742f05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:21.740573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:21.741443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:21.742197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:21.744935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 82/167] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:21.752477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b359692-5a25-4d93-aea7-54b981742f05 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:21.769245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d75c-5dc3-47d9-b247-04a47c9e5b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:21.769876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d75c-5dc3-47d9-b247-04a47c9e5b82 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:21.770074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d75c-5dc3-47d9-b247-04a47c9e5b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:21.770331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d75c-5dc3-47d9-b247-04a47c9e5b82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:21.944421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b359692-5a25-4d93-aea7-54b981742f05 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:21.948248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d75c-5dc3-47d9-b247-04a47c9e5b82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:21.949908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d75c-5dc3-47d9-b247-04a47c9e5b82 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:22.014933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:22.033839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:22.034363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:22.034668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:22.034916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:22.035786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:22.035962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:22.036768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:22.037295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:22.055568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 83/168] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:22.056532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:22.181073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 86/169] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:22.181928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d75c-5dc3-47d9-b247-04a47c9e5b82 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:22.320565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd602781-3ee6-430b-9035-0bca4b5c938a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:22.344739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d75c-5dc3-47d9-b247-04a47c9e5b82 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:22.414124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:22.414745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:22.415031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:22.415309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:22.415990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:22.416295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:22.421219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:22.423179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7384415-08d0-4d1e-b33e-5cb31b9687f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:22.424059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384415-08d0-4d1e-b33e-5cb31b9687f9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:22.424559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384415-08d0-4d1e-b33e-5cb31b9687f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:22.425044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384415-08d0-4d1e-b33e-5cb31b9687f9 None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:22.426418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7384415-08d0-4d1e-b33e-5cb31b9687f9 None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:22.428059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7384415-08d0-4d1e-b33e-5cb31b9687f9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:22.429889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 87/170] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:22 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:22.430876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7384415-08d0-4d1e-b33e-5cb31b9687f9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:22.436464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 84/171] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:22.437791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:22.445526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:22.540868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:22.579430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556bce0-1650-42b4-b76b-08571dc80a1e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:22.616768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84b085-8c26-4f4d-808c-89b649466c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:22.617407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b085-8c26-4f4d-808c-89b649466c81 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:22.617680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b085-8c26-4f4d-808c-89b649466c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:22.617945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84b085-8c26-4f4d-808c-89b649466c81 None admin] PATH_INFO: `/v3/users/832165b8104e4cafa72e1d439a9295d0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:22.619986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84b085-8c26-4f4d-808c-89b649466c81 None admin] RBAC: Authorizing `identity:get_user(user_id=832165b8104e4cafa72e1d439a9295d0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:22.622192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84b085-8c26-4f4d-808c-89b649466c81 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:22.630950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 85/172] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:22 2022] GET /identity/v3/users/832165b8104e4cafa72e1d439a9295d0 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:22.631796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84b085-8c26-4f4d-808c-89b649466c81 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:22.697127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7384415-08d0-4d1e-b33e-5cb31b9687f9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:22.765927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b557978-edc5-4a76-9606-c2173bae4902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:22.766576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b557978-edc5-4a76-9606-c2173bae4902 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:22.766849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b557978-edc5-4a76-9606-c2173bae4902 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:22.767117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b557978-edc5-4a76-9606-c2173bae4902 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:22.770014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b557978-edc5-4a76-9606-c2173bae4902 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:22.771494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b557978-edc5-4a76-9606-c2173bae4902 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:22.773261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 88/173] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:22 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:22.774443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c248c-aeac-426b-9e8d-6eab9546c8f6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:22.782241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c248c-aeac-426b-9e8d-6eab9546c8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:22.782539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c248c-aeac-426b-9e8d-6eab9546c8f6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:22.784038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 89/174] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:22.784821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c248c-aeac-426b-9e8d-6eab9546c8f6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:22.819376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84b085-8c26-4f4d-808c-89b649466c81 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:22.855709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9c413-7c91-40ee-97ef-7311abf75a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:22.856212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9c413-7c91-40ee-97ef-7311abf75a54 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:22.856517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9c413-7c91-40ee-97ef-7311abf75a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:22.856793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9c413-7c91-40ee-97ef-7311abf75a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:22.860781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:22.931111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9c413-7c91-40ee-97ef-7311abf75a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:22.931111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9c413-7c91-40ee-97ef-7311abf75a54 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:22.961190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:23.000815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c248c-aeac-426b-9e8d-6eab9546c8f6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:23.044301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe8020-2348-4597-a7d2-8b78221f0467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:23.044937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8020-2348-4597-a7d2-8b78221f0467 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.045210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8020-2348-4597-a7d2-8b78221f0467 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.045472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8020-2348-4597-a7d2-8b78221f0467 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.064278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 86/175] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:23.065374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9c413-7c91-40ee-97ef-7311abf75a54 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:23.104814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:23.125380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe8020-2348-4597-a7d2-8b78221f0467 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:23.126793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe8020-2348-4597-a7d2-8b78221f0467 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:23.180292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9c413-7c91-40ee-97ef-7311abf75a54 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:23.205383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:23.217987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f19bf4-d504-4dd3-a810-01e4bb47e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:23.218604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f19bf4-d504-4dd3-a810-01e4bb47e2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.218884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f19bf4-d504-4dd3-a810-01e4bb47e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.219160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f19bf4-d504-4dd3-a810-01e4bb47e2c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.220008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f19bf4-d504-4dd3-a810-01e4bb47e2c5 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=57d897223df84e6ba0e4445299c6559e, user.id=910b0103ce4a49babfe0b9390ae4fdd2) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:23.220319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f19bf4-d504-4dd3-a810-01e4bb47e2c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:23.221934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f19bf4-d504-4dd3-a810-01e4bb47e2c5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:23.228471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 87/176] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:23 2022] GET /identity/v3/role_assignments?user.id=910b0103ce4a49babfe0b9390ae4fdd2&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:23.233870 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1463c8-6129-4391-a371-50c4c58f0ae2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.234433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1463c8-6129-4391-a371-50c4c58f0ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.235195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1463c8-6129-4391-a371-50c4c58f0ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.244231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 90/177] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:23.245413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1129d2-71fa-44bc-8e3c-9f876eab6141 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.245696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1129d2-71fa-44bc-8e3c-9f876eab6141 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.246004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1129d2-71fa-44bc-8e3c-9f876eab6141 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.247623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 91/178] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:23.252617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1129d2-71fa-44bc-8e3c-9f876eab6141 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:23.258733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bf1463c8-6129-4391-a371-50c4c58f0ae2 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:23.260354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bf1463c8-6129-4391-a371-50c4c58f0ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:23.364474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 88/179] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:23.364474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593019e8-ed0d-416f-962d-c63d56fd5ded None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.364474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593019e8-ed0d-416f-962d-c63d56fd5ded None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.364474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593019e8-ed0d-416f-962d-c63d56fd5ded None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.364474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 89/180] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:23.364474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593019e8-ed0d-416f-962d-c63d56fd5ded None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:23.386821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1129d2-71fa-44bc-8e3c-9f876eab6141 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:23.410005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:23.441994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb0f98-c58e-4eb5-93b2-51bf7a730e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:23.443467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0f98-c58e-4eb5-93b2-51bf7a730e02 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.444052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0f98-c58e-4eb5-93b2-51bf7a730e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.444584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb0f98-c58e-4eb5-93b2-51bf7a730e02 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.446598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb0f98-c58e-4eb5-93b2-51bf7a730e02 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:23.448212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb0f98-c58e-4eb5-93b2-51bf7a730e02 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:23.450494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 92/181] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:23 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:23.452067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce020f-1c74-4789-a6b0-1fafcb194bc6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.452067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce020f-1c74-4789-a6b0-1fafcb194bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.452268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce020f-1c74-4789-a6b0-1fafcb194bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.474237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-07ce020f-1c74-4789-a6b0-1fafcb194bc6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:23.476558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07ce020f-1c74-4789-a6b0-1fafcb194bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:23.522145 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:23.528666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593019e8-ed0d-416f-962d-c63d56fd5ded None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:23.562547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 93/182] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:23.565349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03258e85-938e-4ea8-8f6e-7a79f5c25fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:23.565984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258e85-938e-4ea8-8f6e-7a79f5c25fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.566260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258e85-938e-4ea8-8f6e-7a79f5c25fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.566535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258e85-938e-4ea8-8f6e-7a79f5c25fe4 None admin] PATH_INFO: `/v3/projects/96d24afaa5cb4cb496972b69a68298ed` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.567636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03258e85-938e-4ea8-8f6e-7a79f5c25fe4 None admin] RBAC: Authorizing `identity:get_project(project_id=96d24afaa5cb4cb496972b69a68298ed)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:23.569150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03258e85-938e-4ea8-8f6e-7a79f5c25fe4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:23.570004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f21bbb-b33a-4812-8b7e-dbf814ebf0c2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.570525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f21bbb-b33a-4812-8b7e-dbf814ebf0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.570946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 90/183] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:23 2022] GET /identity/v3/projects/96d24afaa5cb4cb496972b69a68298ed => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:23.571492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03258e85-938e-4ea8-8f6e-7a79f5c25fe4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:23.571738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f21bbb-b33a-4812-8b7e-dbf814ebf0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.594356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-70f21bbb-b33a-4812-8b7e-dbf814ebf0c2 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:23.595821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-70f21bbb-b33a-4812-8b7e-dbf814ebf0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:23.664575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:23.672644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 94/184] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:23.673523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f21bbb-b33a-4812-8b7e-dbf814ebf0c2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:23.689542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03258e85-938e-4ea8-8f6e-7a79f5c25fe4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:23.727606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb5ec3-1a48-46b4-aac2-bce24bb00b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:23.728251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5ec3-1a48-46b4-aac2-bce24bb00b68 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.728560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5ec3-1a48-46b4-aac2-bce24bb00b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.728835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5ec3-1a48-46b4-aac2-bce24bb00b68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.739733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb5ec3-1a48-46b4-aac2-bce24bb00b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-bceb5ec3-1a48-46b4-aac2-bce24bb00b68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:23.744403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:23.745307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:23.746470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 91/185] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:23.747245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb5ec3-1a48-46b4-aac2-bce24bb00b68 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:23.784797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:23.788121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f21bbb-b33a-4812-8b7e-dbf814ebf0c2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:23.824298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:23.824929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.825208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.825476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.826192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] RBAC: Adding query filter params (role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=57d897223df84e6ba0e4445299c6559e, user.id=832165b8104e4cafa72e1d439a9295d0) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:23.826494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:23.827912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:23.833448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 95/186] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:23 2022] GET /identity/v3/role_assignments?user.id=832165b8104e4cafa72e1d439a9295d0&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:23.834206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:23.848254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb5ec3-1a48-46b4-aac2-bce24bb00b68 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:23.892999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:23.893633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:23.894360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:23.894640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:23.895722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] RBAC: Adding query filter params (role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=96d24afaa5cb4cb496972b69a68298ed, user.id=832165b8104e4cafa72e1d439a9295d0) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:23.896040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:23.897544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:23.905107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 92/187] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:23 2022] GET /identity/v3/role_assignments?user.id=832165b8104e4cafa72e1d439a9295d0&scope.project.id=96d24afaa5cb4cb496972b69a68298ed&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:23.905990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:23.921740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:23.987934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44efdee7-b228-4e55-9aa1-c4738166023c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.034632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8558587-9732-4d43-a1ef-dabf44239927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.035292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8558587-9732-4d43-a1ef-dabf44239927 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.035570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8558587-9732-4d43-a1ef-dabf44239927 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.035844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8558587-9732-4d43-a1ef-dabf44239927 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.037687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8558587-9732-4d43-a1ef-dabf44239927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:24.039497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8558587-9732-4d43-a1ef-dabf44239927 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:24.044535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 96/188] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:24.044535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8901f-91ea-4ed1-b0a9-d378fab6cd97 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.044535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8901f-91ea-4ed1-b0a9-d378fab6cd97 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.044535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8901f-91ea-4ed1-b0a9-d378fab6cd97 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.045828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 97/189] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:24.046582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8901f-91ea-4ed1-b0a9-d378fab6cd97 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.065793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a849920-2662-4b48-9af4-31b8c51c2bc3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.088913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:24.109274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4350b-2ab2-485b-abe7-262eee6af959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.109922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4350b-2ab2-485b-abe7-262eee6af959 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.110206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4350b-2ab2-485b-abe7-262eee6af959 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.110487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4350b-2ab2-485b-abe7-262eee6af959 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.123248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4350b-2ab2-485b-abe7-262eee6af959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-cac4350b-2ab2-485b-abe7-262eee6af959 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:24.128569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:24.129435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:24.130204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 93/190] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:24.130834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4350b-2ab2-485b-abe7-262eee6af959 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.180684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8901f-91ea-4ed1-b0a9-d378fab6cd97 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.204420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:24.214364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2e6e7-a60b-4673-9ca0-9caf92f69c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.215577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2e6e7-a60b-4673-9ca0-9caf92f69c35 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.215847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2e6e7-a60b-4673-9ca0-9caf92f69c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.216121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2e6e7-a60b-4673-9ca0-9caf92f69c35 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.216803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2e6e7-a60b-4673-9ca0-9caf92f69c35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:24.217102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2e6e7-a60b-4673-9ca0-9caf92f69c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:24.218473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2e6e7-a60b-4673-9ca0-9caf92f69c35 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:24.226667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 98/191] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:24.227858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19367ff1-df54-419d-b56b-53cd87fd3cc1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.228135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19367ff1-df54-419d-b56b-53cd87fd3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.228406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19367ff1-df54-419d-b56b-53cd87fd3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.237347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4350b-2ab2-485b-abe7-262eee6af959 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.245795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-19367ff1-df54-419d-b56b-53cd87fd3cc1 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:24.247199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-19367ff1-df54-419d-b56b-53cd87fd3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:24.271365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ee3ea-6bda-4d19-a4e1-fa815b4f3733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.271977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ee3ea-6bda-4d19-a4e1-fa815b4f3733 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.272256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ee3ea-6bda-4d19-a4e1-fa815b4f3733 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.272560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ee3ea-6bda-4d19-a4e1-fa815b4f3733 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.273202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ee3ea-6bda-4d19-a4e1-fa815b4f3733 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:24.274626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ee3ea-6bda-4d19-a4e1-fa815b4f3733 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:24.294750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 94/192] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:00:24 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:24.295557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ee3ea-6bda-4d19-a4e1-fa815b4f3733 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.322652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 99/193] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:24.322652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19367ff1-df54-419d-b56b-53cd87fd3cc1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.360349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:24.441396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ee3ea-6bda-4d19-a4e1-fa815b4f3733 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.452429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19367ff1-df54-419d-b56b-53cd87fd3cc1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.470992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:24.485807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.486432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.487245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.487245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.487919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:24.488108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:24.489522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:24.495309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7504f8-ef48-441a-a07b-c837e29deb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.496538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7504f8-ef48-441a-a07b-c837e29deb91 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.496817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7504f8-ef48-441a-a07b-c837e29deb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.497081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7504f8-ef48-441a-a07b-c837e29deb91 None admin] PATH_INFO: `/v3/users/10370723b3eb4c20a1f90fdf745ac0ef` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.499020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 95/194] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:24.499837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.512352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7504f8-ef48-441a-a07b-c837e29deb91 None admin] RBAC: Authorizing `identity:get_user(user_id=10370723b3eb4c20a1f90fdf745ac0ef)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:24.513890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7504f8-ef48-441a-a07b-c837e29deb91 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:24.515585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 100/195] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:24 2022] GET /identity/v3/users/10370723b3eb4c20a1f90fdf745ac0ef => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:24.516466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7504f8-ef48-441a-a07b-c837e29deb91 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.623702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:24.634975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914f6a1-ac10-42b3-af38-c9fc31131540 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.646201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7504f8-ef48-441a-a07b-c837e29deb91 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.678540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf8b6c-9423-47e4-ac69-443dd066cb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.679183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf8b6c-9423-47e4-ac69-443dd066cb49 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.679462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf8b6c-9423-47e4-ac69-443dd066cb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.679731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf8b6c-9423-47e4-ac69-443dd066cb49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.697005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf8b6c-9423-47e4-ac69-443dd066cb49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:24.701337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d9033-bc50-46a5-a3e1-1649f8e69742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.701977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d9033-bc50-46a5-a3e1-1649f8e69742 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.702254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d9033-bc50-46a5-a3e1-1649f8e69742 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b0cf8b6c-9423-47e4-ac69-443dd066cb49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:24.702372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:24.703179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:24.704027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:24.704638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d9033-bc50-46a5-a3e1-1649f8e69742 None admin] PATH_INFO: `/v3/users/10370723b3eb4c20a1f90fdf745ac0ef` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.704705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 96/196] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:24.704705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf8b6c-9423-47e4-ac69-443dd066cb49 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.705917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d9033-bc50-46a5-a3e1-1649f8e69742 None admin] RBAC: Authorizing `identity:get_user(user_id=10370723b3eb4c20a1f90fdf745ac0ef)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:24.707435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d9033-bc50-46a5-a3e1-1649f8e69742 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:24.709595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 101/197] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:24 2022] GET /identity/v3/users/10370723b3eb4c20a1f90fdf745ac0ef => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:24.713358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d9033-bc50-46a5-a3e1-1649f8e69742 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.764916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:24.823653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf8b6c-9423-47e4-ac69-443dd066cb49 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.856037 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d9033-bc50-46a5-a3e1-1649f8e69742 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:24.868951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a83e53-ac8a-4523-a1ea-f2c91ec31078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.869615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83e53-ac8a-4523-a1ea-f2c91ec31078 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.869901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83e53-ac8a-4523-a1ea-f2c91ec31078 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.870433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a83e53-ac8a-4523-a1ea-f2c91ec31078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.893039 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:24.893745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:24.894095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:24.894438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:24.895177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:24.895550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:24.897036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:24.906939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 102/198] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:24.907713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:24.924290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:24.966693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a83e53-ac8a-4523-a1ea-f2c91ec31078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:24.969005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a83e53-ac8a-4523-a1ea-f2c91ec31078 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.028599 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3659ee-9f23-428e-93fe-06eb7da3d0f9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:25.035511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:25.071856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79c5b-756f-4dbb-81bb-b768a48c806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:25.072728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79c5b-756f-4dbb-81bb-b768a48c806a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:25.073112 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79c5b-756f-4dbb-81bb-b768a48c806a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:25.073488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79c5b-756f-4dbb-81bb-b768a48c806a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:25.102365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 97/199] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:25.103427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a83e53-ac8a-4523-a1ea-f2c91ec31078 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:25.148758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79c5b-756f-4dbb-81bb-b768a48c806a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:25.150512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79c5b-756f-4dbb-81bb-b768a48c806a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.167709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:25.217013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a83e53-ac8a-4523-a1ea-f2c91ec31078 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:25.254300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f30c94-6451-4962-8386-04134d1eb9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:25.255260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f30c94-6451-4962-8386-04134d1eb9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:25.255731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f30c94-6451-4962-8386-04134d1eb9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:25.256254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f30c94-6451-4962-8386-04134d1eb9b7 None admin] PATH_INFO: `/v3/users/74fa483d9f204772b821eba2b9690718` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:25.257720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f30c94-6451-4962-8386-04134d1eb9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=74fa483d9f204772b821eba2b9690718)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:25.259641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f30c94-6451-4962-8386-04134d1eb9b7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.261589 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 98/200] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:25 2022] GET /identity/v3/users/74fa483d9f204772b821eba2b9690718 => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:25.262653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f30c94-6451-4962-8386-04134d1eb9b7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:25.267080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 103/201] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:25.267080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79c5b-756f-4dbb-81bb-b768a48c806a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:25.282357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:25.373929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f30c94-6451-4962-8386-04134d1eb9b7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:25.377333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79c5b-756f-4dbb-81bb-b768a48c806a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:25.391401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:25.414192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466b911-981e-4fc7-88e4-db18beb47b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:25.415146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e2253-fb07-4af7-99d4-2af40bbd7beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:25.415506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466b911-981e-4fc7-88e4-db18beb47b7e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:25.415750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e2253-fb07-4af7-99d4-2af40bbd7beb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:25.416087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e2253-fb07-4af7-99d4-2af40bbd7beb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:25.416364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e2253-fb07-4af7-99d4-2af40bbd7beb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:25.416594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466b911-981e-4fc7-88e4-db18beb47b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:25.417057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466b911-981e-4fc7-88e4-db18beb47b7e None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:25.419222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466b911-981e-4fc7-88e4-db18beb47b7e None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:25.421059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466b911-981e-4fc7-88e4-db18beb47b7e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.423178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 99/202] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:25 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:25.424138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466b911-981e-4fc7-88e4-db18beb47b7e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:25.490368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e2253-fb07-4af7-99d4-2af40bbd7beb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:25.491910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e2253-fb07-4af7-99d4-2af40bbd7beb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.519099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:25.537508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466b911-981e-4fc7-88e4-db18beb47b7e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:25.575158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f0e96-23e2-4a4a-97a9-897de6999a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:25.576147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f0e96-23e2-4a4a-97a9-897de6999a80 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:25.576767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f0e96-23e2-4a4a-97a9-897de6999a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:25.577234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f0e96-23e2-4a4a-97a9-897de6999a80 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:25.578582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f0e96-23e2-4a4a-97a9-897de6999a80 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:25.580316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f0e96-23e2-4a4a-97a9-897de6999a80 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.582255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 100/203] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:25 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:25.583350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f0e96-23e2-4a4a-97a9-897de6999a80 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:25.607150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 104/204] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:25.608081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e2253-fb07-4af7-99d4-2af40bbd7beb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:25.626716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:25.697901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f0e96-23e2-4a4a-97a9-897de6999a80 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:25.725549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e2253-fb07-4af7-99d4-2af40bbd7beb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:25.738522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:25.739493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:25.739961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:25.740629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:25.741658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=a74bec509592469588b2a87649030293, user.id=10370723b3eb4c20a1f90fdf745ac0ef) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:25.742153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:25.743839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.750515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 101/205] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:25 2022] GET /identity/v3/role_assignments?user.id=10370723b3eb4c20a1f90fdf745ac0ef&scope.project.id=a74bec509592469588b2a87649030293&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:25.751742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:25.763795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:25.764465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:25.764775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:25.765044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:25.766184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] RBAC: Adding query filter params (role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=a74bec509592469588b2a87649030293, user.id=10370723b3eb4c20a1f90fdf745ac0ef) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:25.766184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:25.767709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.774484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:25.774623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 105/206] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:25 2022] GET /identity/v3/role_assignments?user.id=10370723b3eb4c20a1f90fdf745ac0ef&scope.project.id=a74bec509592469588b2a87649030293&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:25.877936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5eb48-38e5-4011-8290-eb4f83270886 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:25.918449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb0299-0247-42f2-97df-21f6dbb1bab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:25.919345 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb0299-0247-42f2-97df-21f6dbb1bab0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:25.919759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb0299-0247-42f2-97df-21f6dbb1bab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:25.920178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb0299-0247-42f2-97df-21f6dbb1bab0 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:25.921361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb0299-0247-42f2-97df-21f6dbb1bab0 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:25.922840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb0299-0247-42f2-97df-21f6dbb1bab0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:25.924724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 102/207] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:25 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:25.930770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:26.019512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:26.048119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef529-a4c0-400d-8779-7619a55e8228 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:26.086710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e190251-cbfd-4c25-ba02-0d6a4ae554a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:26.087323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190251-cbfd-4c25-ba02-0d6a4ae554a2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:26.087610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190251-cbfd-4c25-ba02-0d6a4ae554a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:26.087909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190251-cbfd-4c25-ba02-0d6a4ae554a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:26.088653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e190251-cbfd-4c25-ba02-0d6a4ae554a2 None admin] RBAC: Adding query filter params (role.id=585290ba3e014e9e9ae58a7c1d1f1f67, scope.project.id=57d897223df84e6ba0e4445299c6559e, user.id=74fa483d9f204772b821eba2b9690718) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:26.088998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e190251-cbfd-4c25-ba02-0d6a4ae554a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:26.091103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e190251-cbfd-4c25-ba02-0d6a4ae554a2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:26.097351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 106/208] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:25 2022] GET /identity/v3/role_assignments?user.id=74fa483d9f204772b821eba2b9690718&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=585290ba3e014e9e9ae58a7c1d1f1f67 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:26.638190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b298-93d6-476b-8e50-1b01eb422fce None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:26.638725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b298-93d6-476b-8e50-1b01eb422fce None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:26.638801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b298-93d6-476b-8e50-1b01eb422fce None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:26.640164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 103/209] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:26.646080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd238d7-e845-443f-a611-74a932bddf3e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:26.646371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd238d7-e845-443f-a611-74a932bddf3e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:26.646652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd238d7-e845-443f-a611-74a932bddf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:26.670329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4bd238d7-e845-443f-a611-74a932bddf3e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:26.672113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4bd238d7-e845-443f-a611-74a932bddf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:26.677370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be515b8-ddc7-4d71-b5ea-7de45f61604d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:26.677642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be515b8-ddc7-4d71-b5ea-7de45f61604d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:26.677922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be515b8-ddc7-4d71-b5ea-7de45f61604d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:26.679594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 104/210] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:26.686020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22debfce-3a81-4ecc-a6a6-220a120cfbbd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:26.686600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22debfce-3a81-4ecc-a6a6-220a120cfbbd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:26.686895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22debfce-3a81-4ecc-a6a6-220a120cfbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:26.706308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-22debfce-3a81-4ecc-a6a6-220a120cfbbd None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:26.707603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-22debfce-3a81-4ecc-a6a6-220a120cfbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:26.749442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 107/211] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:26.774707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd238d7-e845-443f-a611-74a932bddf3e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:26.781143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 105/212] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:26.812301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22debfce-3a81-4ecc-a6a6-220a120cfbbd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:26.827687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:26.848798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:26.889391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd238d7-e845-443f-a611-74a932bddf3e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:26.917527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22debfce-3a81-4ecc-a6a6-220a120cfbbd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:26.937750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a19ea-9f6e-4ea9-a615-749e9045f3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:26.938385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a19ea-9f6e-4ea9-a615-749e9045f3cd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:26.938657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a19ea-9f6e-4ea9-a615-749e9045f3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:26.938928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a19ea-9f6e-4ea9-a615-749e9045f3cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:26.952169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a19ea-9f6e-4ea9-a615-749e9045f3cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ff1a19ea-9f6e-4ea9-a615-749e9045f3cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:26.958383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:26.959249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:26.960388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 108/213] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:26.966426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a19ea-9f6e-4ea9-a615-749e9045f3cd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:26.970338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2461071-57af-4647-8a8c-4788340897b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:26.970978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2461071-57af-4647-8a8c-4788340897b0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:26.971257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2461071-57af-4647-8a8c-4788340897b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:26.971527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2461071-57af-4647-8a8c-4788340897b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:26.972711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2461071-57af-4647-8a8c-4788340897b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:26.974148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2461071-57af-4647-8a8c-4788340897b0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:26.980226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 106/214] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:26.981952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2461071-57af-4647-8a8c-4788340897b0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:27.067721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:27.089754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2461071-57af-4647-8a8c-4788340897b0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:27.094158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a19ea-9f6e-4ea9-a615-749e9045f3cd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:27.126896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8c02-c27e-4dc2-b63d-2f3741ed3589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:27.127678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8c02-c27e-4dc2-b63d-2f3741ed3589 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:27.128057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8c02-c27e-4dc2-b63d-2f3741ed3589 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:27.128429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8c02-c27e-4dc2-b63d-2f3741ed3589 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:27.129215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c8c02-c27e-4dc2-b63d-2f3741ed3589 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:27.130746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c8c02-c27e-4dc2-b63d-2f3741ed3589 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:27.131078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:27.137512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:27.138335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:27.138753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:27.139164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:27.140278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:27.140749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:27.143658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:27.154849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 109/215] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:27.164551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 107/216] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:00:26 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:27.165185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:27.290129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783ce65-d904-4074-8b7d-019e271d0873 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:27.328306 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe9873-5228-41af-9c94-bbcfc885f6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:27.328967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe9873-5228-41af-9c94-bbcfc885f6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:27.329241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe9873-5228-41af-9c94-bbcfc885f6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:27.329567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe9873-5228-41af-9c94-bbcfc885f6e9 None admin] PATH_INFO: `/v3/users/b7ef258e77e3400c98dc630ac338bdeb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:27.345975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe9873-5228-41af-9c94-bbcfc885f6e9 None admin] RBAC: Authorizing `identity:get_user(user_id=b7ef258e77e3400c98dc630ac338bdeb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:27.347440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe9873-5228-41af-9c94-bbcfc885f6e9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:27.349459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 110/217] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:27 2022] GET /identity/v3/users/b7ef258e77e3400c98dc630ac338bdeb => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:27.354633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8c02-c27e-4dc2-b63d-2f3741ed3589 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:27.484300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8c02-c27e-4dc2-b63d-2f3741ed3589 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:27.524002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b65c38-5a74-4dc6-ad57-0bfd1f79bb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:27.525043 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b65c38-5a74-4dc6-ad57-0bfd1f79bb98 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:27.525548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b65c38-5a74-4dc6-ad57-0bfd1f79bb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:27.526064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b65c38-5a74-4dc6-ad57-0bfd1f79bb98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:27.537214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:27.609796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b65c38-5a74-4dc6-ad57-0bfd1f79bb98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:27.611693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b65c38-5a74-4dc6-ad57-0bfd1f79bb98 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:27.729179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 108/218] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:27.735778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe9873-5228-41af-9c94-bbcfc885f6e9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:27.808999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:27.875770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe9873-5228-41af-9c94-bbcfc885f6e9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:27.927581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5b34d-5ffe-4791-923e-3142a702cc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:27.928606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b34d-5ffe-4791-923e-3142a702cc31 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:27.929086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b34d-5ffe-4791-923e-3142a702cc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:27.929556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5b34d-5ffe-4791-923e-3142a702cc31 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:27.930925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5b34d-5ffe-4791-923e-3142a702cc31 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:27.932669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5b34d-5ffe-4791-923e-3142a702cc31 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:27.935815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 111/219] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:27 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:27.941724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b65c38-5a74-4dc6-ad57-0bfd1f79bb98 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:27.995275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:28.078679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b65c38-5a74-4dc6-ad57-0bfd1f79bb98 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:28.095769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da85c5a-c410-4dc6-b1e7-5c9f75be7d3e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.096729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da85c5a-c410-4dc6-b1e7-5c9f75be7d3e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.097214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da85c5a-c410-4dc6-b1e7-5c9f75be7d3e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.098633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 112/220] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:28.107315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e528-c260-460a-acc7-c44125157f1e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.107669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e528-c260-460a-acc7-c44125157f1e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.108038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1e528-c260-460a-acc7-c44125157f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.124577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cf40d-2318-4efa-8dc5-4da571e5f367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:28.125479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cf40d-2318-4efa-8dc5-4da571e5f367 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.125988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cf40d-2318-4efa-8dc5-4da571e5f367 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.126451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cf40d-2318-4efa-8dc5-4da571e5f367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.127354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cf40d-2318-4efa-8dc5-4da571e5f367 None admin] RBAC: Adding query filter params (role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=a74bec509592469588b2a87649030293, user.id=b7ef258e77e3400c98dc630ac338bdeb) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:28.127833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cf40d-2318-4efa-8dc5-4da571e5f367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:28.129455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cf40d-2318-4efa-8dc5-4da571e5f367 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:28.135518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 109/221] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:27 2022] GET /identity/v3/role_assignments?user.id=b7ef258e77e3400c98dc630ac338bdeb&scope.project.id=a74bec509592469588b2a87649030293&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:28.140582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3ac1e528-c260-460a-acc7-c44125157f1e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:28.142048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3ac1e528-c260-460a-acc7-c44125157f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:28.245243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 113/222] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:28.257106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02fa04-7725-4956-9ace-9bba241c9495 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.257756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02fa04-7725-4956-9ace-9bba241c9495 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.258219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02fa04-7725-4956-9ace-9bba241c9495 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.259769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 110/223] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:28.266150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498e17-5ac0-46b7-857f-e72cdd3bb92b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.266439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498e17-5ac0-46b7-857f-e72cdd3bb92b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.266721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498e17-5ac0-46b7-857f-e72cdd3bb92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.273581 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02fa04-7725-4956-9ace-9bba241c9495 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:28.290180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4f498e17-5ac0-46b7-857f-e72cdd3bb92b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:28.291390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4f498e17-5ac0-46b7-857f-e72cdd3bb92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:28.361850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:28.404799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 114/224] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:28.424417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498e17-5ac0-46b7-857f-e72cdd3bb92b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:28.446746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02fa04-7725-4956-9ace-9bba241c9495 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:28.451856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:28.523856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db6d2c-c85c-42a0-8cae-6a5386866536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:28.524533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db6d2c-c85c-42a0-8cae-6a5386866536 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.524818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db6d2c-c85c-42a0-8cae-6a5386866536 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.525090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db6d2c-c85c-42a0-8cae-6a5386866536 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.545403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db6d2c-c85c-42a0-8cae-6a5386866536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f5db6d2c-c85c-42a0-8cae-6a5386866536 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:28.557909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:28.559392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:28.560158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:28.561046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 111/225] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:28.566374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db6d2c-c85c-42a0-8cae-6a5386866536 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:28.594872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498e17-5ac0-46b7-857f-e72cdd3bb92b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:28.670351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db6d2c-c85c-42a0-8cae-6a5386866536 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:28.709623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:28.710297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.710585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.710868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.711540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:28.711846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:28.713152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9971d2-9266-40d2-b171-0da3c4fd45be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:28.713353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:28.713804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9971d2-9266-40d2-b171-0da3c4fd45be None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.714085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9971d2-9266-40d2-b171-0da3c4fd45be None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.714362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9971d2-9266-40d2-b171-0da3c4fd45be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.723234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 112/226] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:28.728647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:28.745388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9971d2-9266-40d2-b171-0da3c4fd45be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0d9971d2-9266-40d2-b171-0da3c4fd45be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:28.750195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:28.751070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:28.751827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 115/227] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:28.762073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9971d2-9266-40d2-b171-0da3c4fd45be None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:28.780811 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:28.865410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e7b17-a4d6-43f8-91fd-3e8935c471a0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:28.867909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:28.937590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8916b-0011-4b86-95f8-04c561d8483a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:28.938582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8916b-0011-4b86-95f8-04c561d8483a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:28.939059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8916b-0011-4b86-95f8-04c561d8483a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:28.939526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8916b-0011-4b86-95f8-04c561d8483a None admin] PATH_INFO: `/v3/users/10370723b3eb4c20a1f90fdf745ac0ef` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:28.946728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8916b-0011-4b86-95f8-04c561d8483a None admin] RBAC: Authorizing `identity:get_user(user_id=10370723b3eb4c20a1f90fdf745ac0ef)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:28.948551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8916b-0011-4b86-95f8-04c561d8483a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:28.950975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 113/228] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:28 2022] GET /identity/v3/users/10370723b3eb4c20a1f90fdf745ac0ef => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:28.958471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8916b-0011-4b86-95f8-04c561d8483a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:28.967245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9971d2-9266-40d2-b171-0da3c4fd45be None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:29.002976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:29.003586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.003879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.004149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.004852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:29.005162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:29.006634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:29.027965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 116/229] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:29.037835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:29.085719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:29.112856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8916b-0011-4b86-95f8-04c561d8483a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:29.156555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be323d-c27a-4323-a35e-508f09d33824 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:29.164634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:29.180541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9dbd7-de33-47fb-a2bd-51c43ccfb590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:29.181189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9dbd7-de33-47fb-a2bd-51c43ccfb590 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.181468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9dbd7-de33-47fb-a2bd-51c43ccfb590 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.181742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9dbd7-de33-47fb-a2bd-51c43ccfb590 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.182867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9dbd7-de33-47fb-a2bd-51c43ccfb590 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:29.184353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9dbd7-de33-47fb-a2bd-51c43ccfb590 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:29.192211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 114/230] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:28 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:29.199114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9dbd7-de33-47fb-a2bd-51c43ccfb590 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:29.201175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed63296-43e1-401e-b98d-f50cd7557fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:29.201830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63296-43e1-401e-b98d-f50cd7557fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.202115 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63296-43e1-401e-b98d-f50cd7557fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.202513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63296-43e1-401e-b98d-f50cd7557fb8 None admin] PATH_INFO: `/v3/users/10370723b3eb4c20a1f90fdf745ac0ef` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.204143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed63296-43e1-401e-b98d-f50cd7557fb8 None admin] RBAC: Authorizing `identity:get_user(user_id=10370723b3eb4c20a1f90fdf745ac0ef)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:29.207054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed63296-43e1-401e-b98d-f50cd7557fb8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:29.209638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 117/231] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:29 2022] GET /identity/v3/users/10370723b3eb4c20a1f90fdf745ac0ef => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:29.215538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed63296-43e1-401e-b98d-f50cd7557fb8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:29.326406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed63296-43e1-401e-b98d-f50cd7557fb8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:29.363781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494081a-fb29-41f6-bc5e-b301a1a23a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:29.364421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494081a-fb29-41f6-bc5e-b301a1a23a04 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.364731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494081a-fb29-41f6-bc5e-b301a1a23a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.365009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494081a-fb29-41f6-bc5e-b301a1a23a04 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.366541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494081a-fb29-41f6-bc5e-b301a1a23a04 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:29.368000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494081a-fb29-41f6-bc5e-b301a1a23a04 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:29.369979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 118/232] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:29 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:29.376761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494081a-fb29-41f6-bc5e-b301a1a23a04 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:29.422011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:29.494067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494081a-fb29-41f6-bc5e-b301a1a23a04 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:29.519496 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9dbd7-de33-47fb-a2bd-51c43ccfb590 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:29.560709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebabd9c-87f5-430b-9f92-4059ddbf0c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:29.561540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebabd9c-87f5-430b-9f92-4059ddbf0c68 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.561882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebabd9c-87f5-430b-9f92-4059ddbf0c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.562161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebabd9c-87f5-430b-9f92-4059ddbf0c68 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293/users/10370723b3eb4c20a1f90fdf745ac0ef/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.564161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebabd9c-87f5-430b-9f92-4059ddbf0c68 None admin] RBAC: Authorizing `identity:create_grant(project_id=a74bec509592469588b2a87649030293, user_id=10370723b3eb4c20a1f90fdf745ac0ef, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:29.565803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebabd9c-87f5-430b-9f92-4059ddbf0c68 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:29.572731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286a160-a0ca-4cc6-a5d2-10d9f22a666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:29.573665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a160-a0ca-4cc6-a5d2-10d9f22a666c None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.574031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a160-a0ca-4cc6-a5d2-10d9f22a666c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.574303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a160-a0ca-4cc6-a5d2-10d9f22a666c None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293/users/10370723b3eb4c20a1f90fdf745ac0ef/roles/199fa373d2ff46b7a0c430b4a8c056e9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.580758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286a160-a0ca-4cc6-a5d2-10d9f22a666c None admin] RBAC: Authorizing `identity:create_grant(project_id=a74bec509592469588b2a87649030293, user_id=10370723b3eb4c20a1f90fdf745ac0ef, role_id=199fa373d2ff46b7a0c430b4a8c056e9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:29.581560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 115/233] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:29 2022] PUT /identity/v3/projects/a74bec509592469588b2a87649030293/users/10370723b3eb4c20a1f90fdf745ac0ef/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:29.586887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286a160-a0ca-4cc6-a5d2-10d9f22a666c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:29.617000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 119/234] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:29 2022] PUT /identity/v3/projects/a74bec509592469588b2a87649030293/users/10370723b3eb4c20a1f90fdf745ac0ef/roles/199fa373d2ff46b7a0c430b4a8c056e9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:29.886024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22860-15db-4a77-a8da-487ac88d3f50 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.887090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22860-15db-4a77-a8da-487ac88d3f50 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.887315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22860-15db-4a77-a8da-487ac88d3f50 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.889210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 116/235] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:29.894881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5455-afe2-42e3-9152-57422b7cbf0d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.895167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5455-afe2-42e3-9152-57422b7cbf0d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.895445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5455-afe2-42e3-9152-57422b7cbf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.915069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-060d5455-afe2-42e3-9152-57422b7cbf0d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:29.916258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-060d5455-afe2-42e3-9152-57422b7cbf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:29.955909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34065a4d-6595-457e-9247-82a97647dc6d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.956203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34065a4d-6595-457e-9247-82a97647dc6d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.956515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34065a4d-6595-457e-9247-82a97647dc6d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.958035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 117/236] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:29.966129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c7eef-dab4-410f-ad20-4a8666bf04cf None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:29.967020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c7eef-dab4-410f-ad20-4a8666bf04cf None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:29.967399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c7eef-dab4-410f-ad20-4a8666bf04cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:29.995603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-362c7eef-dab4-410f-ad20-4a8666bf04cf None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:29.998228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-362c7eef-dab4-410f-ad20-4a8666bf04cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:30.005664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 120/237] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:30.015888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:30.034293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5455-afe2-42e3-9152-57422b7cbf0d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:30.068055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:30.112974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 118/238] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:30.141606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c7eef-dab4-410f-ad20-4a8666bf04cf None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:30.147018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5455-afe2-42e3-9152-57422b7cbf0d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:30.183427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58454-be95-4298-b5a0-a7aa1515c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:30.184339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58454-be95-4298-b5a0-a7aa1515c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:30.184423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58454-be95-4298-b5a0-a7aa1515c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:30.184715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58454-be95-4298-b5a0-a7aa1515c0b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:30.205591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b58454-be95-4298-b5a0-a7aa1515c0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d0b58454-be95-4298-b5a0-a7aa1515c0b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:30.212190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:30.213097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:30.213865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:30.216881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 121/239] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:30.223160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58454-be95-4298-b5a0-a7aa1515c0b8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:30.328370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c7eef-dab4-410f-ad20-4a8666bf04cf None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:30.332325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:30.362584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58454-be95-4298-b5a0-a7aa1515c0b8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:30.398284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ddbad-f949-4412-bc01-130d76e0a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:30.400885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ddbad-f949-4412-bc01-130d76e0a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:30.403446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ddbad-f949-4412-bc01-130d76e0a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:30.403940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ddbad-f949-4412-bc01-130d76e0a1c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:30.405411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ddbad-f949-4412-bc01-130d76e0a1c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:30.407330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ddbad-f949-4412-bc01-130d76e0a1c0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:30.412963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 119/240] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:30.431054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ddbad-f949-4412-bc01-130d76e0a1c0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:30.451520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:30.452239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:30.452537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:30.452796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:30.453316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:30.453488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:30.453808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:30.455192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:30.476043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 122/241] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:30.482350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:30.552340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ddbad-f949-4412-bc01-130d76e0a1c0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:30.603443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db220fb1-8aa0-4d58-95ae-1d1d2bbcbe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:30.604063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db220fb1-8aa0-4d58-95ae-1d1d2bbcbe19 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:30.604337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db220fb1-8aa0-4d58-95ae-1d1d2bbcbe19 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:30.605975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db220fb1-8aa0-4d58-95ae-1d1d2bbcbe19 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:30.606631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db220fb1-8aa0-4d58-95ae-1d1d2bbcbe19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:30.608067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db220fb1-8aa0-4d58-95ae-1d1d2bbcbe19 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:30.648663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 120/242] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:00:30 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:00:30.677335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8562497-bb5c-492c-b750-978ce2440e81 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:30.725796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a385a-3547-435f-9c7c-ff89658b2bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:30.726480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a385a-3547-435f-9c7c-ff89658b2bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:30.726755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a385a-3547-435f-9c7c-ff89658b2bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:30.727022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a385a-3547-435f-9c7c-ff89658b2bb6 None admin] PATH_INFO: `/v3/users/c80bc1629fea4840bcb72d175ac1acf1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:30.747222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a385a-3547-435f-9c7c-ff89658b2bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=c80bc1629fea4840bcb72d175ac1acf1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:30.748747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a385a-3547-435f-9c7c-ff89658b2bb6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:30.750558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 123/243] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:30 2022] GET /identity/v3/users/c80bc1629fea4840bcb72d175ac1acf1 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:30.756487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db220fb1-8aa0-4d58-95ae-1d1d2bbcbe19 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:30.843805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2544e3-9a92-4281-b6c2-04983ba73358 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:30.844503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2544e3-9a92-4281-b6c2-04983ba73358 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:30.844984 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2544e3-9a92-4281-b6c2-04983ba73358 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:30.846921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 124/244] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:30.853395 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb6a4c-e4ac-4e09-ada3-9730be08052b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:30.853969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb6a4c-e4ac-4e09-ada3-9730be08052b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:30.854584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb6a4c-e4ac-4e09-ada3-9730be08052b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:30.863068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:30.871987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:30.883768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-38eb6a4c-e4ac-4e09-ada3-9730be08052b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:30.885859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-38eb6a4c-e4ac-4e09-ada3-9730be08052b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:30.910638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db220fb1-8aa0-4d58-95ae-1d1d2bbcbe19 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:30.975407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 125/245] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:30.988992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7990e-a7bb-4264-b9a4-f4e434e9f389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:30.989624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7990e-a7bb-4264-b9a4-f4e434e9f389 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:30.989905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7990e-a7bb-4264-b9a4-f4e434e9f389 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:30.990180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7990e-a7bb-4264-b9a4-f4e434e9f389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:31.005922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb6a4c-e4ac-4e09-ada3-9730be08052b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:31.086338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7990e-a7bb-4264-b9a4-f4e434e9f389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:31.088382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7990e-a7bb-4264-b9a4-f4e434e9f389 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:31.144755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb6a4c-e4ac-4e09-ada3-9730be08052b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:31.192633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be665ff5-6494-4a2f-8e55-57b6a9caa4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:31.193423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:31.194581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be665ff5-6494-4a2f-8e55-57b6a9caa4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:31.195100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be665ff5-6494-4a2f-8e55-57b6a9caa4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:31.195586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be665ff5-6494-4a2f-8e55-57b6a9caa4a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:31.199769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:31.216558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be665ff5-6494-4a2f-8e55-57b6a9caa4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-be665ff5-6494-4a2f-8e55-57b6a9caa4a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:31.222280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:31.223173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:31.224024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:31.227992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 126/246] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:31.232523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 121/247] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:31.235082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be665ff5-6494-4a2f-8e55-57b6a9caa4a1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:31.235220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7990e-a7bb-4264-b9a4-f4e434e9f389 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:31.365110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be665ff5-6494-4a2f-8e55-57b6a9caa4a1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:31.385570 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7990e-a7bb-4264-b9a4-f4e434e9f389 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:31.425260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320a9bc-1947-4694-b37c-d16380efab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:31.426272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320a9bc-1947-4694-b37c-d16380efab3b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:31.427221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320a9bc-1947-4694-b37c-d16380efab3b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:31.427503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320a9bc-1947-4694-b37c-d16380efab3b None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:31.429056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320a9bc-1947-4694-b37c-d16380efab3b None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:31.431429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320a9bc-1947-4694-b37c-d16380efab3b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:31.434293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 127/248] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:31 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:31.437444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:31.438786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:31.439276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:31.439748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:31.440783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:31.440986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320a9bc-1947-4694-b37c-d16380efab3b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:31.441530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:31.443364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:31.454595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 122/249] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:31.467246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:31.524295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:31.538362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:31.601462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320a9bc-1947-4694-b37c-d16380efab3b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:31.658605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67470387-8577-490e-a1e7-e4f31dd8d0b3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:31.663369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:31.663994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:31.664270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:31.664624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:31.665362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] RBAC: Adding query filter params (role.id=585290ba3e014e9e9ae58a7c1d1f1f67, scope.project.id=a74bec509592469588b2a87649030293, user.id=c80bc1629fea4840bcb72d175ac1acf1) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:31.665668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:31.667202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:31.684801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 128/250] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:31 2022] GET /identity/v3/role_assignments?user.id=c80bc1629fea4840bcb72d175ac1acf1&scope.project.id=a74bec509592469588b2a87649030293&role.id=585290ba3e014e9e9ae58a7c1d1f1f67 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:31.720350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf40e-409f-44cd-a98a-86f2f3a765e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:31.720940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf40e-409f-44cd-a98a-86f2f3a765e9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:31.721223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf40e-409f-44cd-a98a-86f2f3a765e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:31.721499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bf40e-409f-44cd-a98a-86f2f3a765e9 None admin] PATH_INFO: `/v3/users/b7ef258e77e3400c98dc630ac338bdeb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:31.722786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bf40e-409f-44cd-a98a-86f2f3a765e9 None admin] RBAC: Authorizing `identity:get_user(user_id=b7ef258e77e3400c98dc630ac338bdeb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:31.724231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bf40e-409f-44cd-a98a-86f2f3a765e9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:31.726209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 123/251] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:31 2022] GET /identity/v3/users/b7ef258e77e3400c98dc630ac338bdeb => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:31.732337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:31.889661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238635c-8ad0-4b3a-9a56-51967fcace19 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:31.921358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:31.935283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245e0e-80c8-40d5-89d8-d79746664312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:31.936201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245e0e-80c8-40d5-89d8-d79746664312 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:31.936499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245e0e-80c8-40d5-89d8-d79746664312 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:31.936778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245e0e-80c8-40d5-89d8-d79746664312 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:31.938081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245e0e-80c8-40d5-89d8-d79746664312 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:31.939589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245e0e-80c8-40d5-89d8-d79746664312 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:31.942676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 129/252] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:31 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:31.949141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf40e-409f-44cd-a98a-86f2f3a765e9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:32.082070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bf40e-409f-44cd-a98a-86f2f3a765e9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:32.084528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:32.136116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a8f82-dba2-4cd8-b115-f8468b848e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:32.137425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a8f82-dba2-4cd8-b115-f8468b848e61 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:32.137707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a8f82-dba2-4cd8-b115-f8468b848e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:32.138173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a8f82-dba2-4cd8-b115-f8468b848e61 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293/users/b7ef258e77e3400c98dc630ac338bdeb/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:32.140173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a8f82-dba2-4cd8-b115-f8468b848e61 None admin] RBAC: Authorizing `identity:create_grant(project_id=a74bec509592469588b2a87649030293, user_id=b7ef258e77e3400c98dc630ac338bdeb, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:32.142940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a8f82-dba2-4cd8-b115-f8468b848e61 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:32.165673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 124/253] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:31 2022] PUT /identity/v3/projects/a74bec509592469588b2a87649030293/users/b7ef258e77e3400c98dc630ac338bdeb/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:32.510957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d229-838e-4b40-81db-b2e9616bc9a6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:32.512160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d229-838e-4b40-81db-b2e9616bc9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:32.512160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28d229-838e-4b40-81db-b2e9616bc9a6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:32.514000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 130/254] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:32.520331 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d4472-3d1b-43d5-a837-98a89833bf54 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:32.520658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d4472-3d1b-43d5-a837-98a89833bf54 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:32.520940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d4472-3d1b-43d5-a837-98a89833bf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:32.549436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-aa6d4472-3d1b-43d5-a837-98a89833bf54 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:32.551585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-aa6d4472-3d1b-43d5-a837-98a89833bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:32.589681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8f72f-39aa-418d-b980-00a14c567323 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:32.590162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8f72f-39aa-418d-b980-00a14c567323 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:32.590567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8f72f-39aa-418d-b980-00a14c567323 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:32.591920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 131/255] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:32.599574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453bf8-c348-4f22-8db3-71267565ca53 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:32.599863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453bf8-c348-4f22-8db3-71267565ca53 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:32.600141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453bf8-c348-4f22-8db3-71267565ca53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:32.637489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f9453bf8-c348-4f22-8db3-71267565ca53 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:32.638697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f9453bf8-c348-4f22-8db3-71267565ca53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:32.689768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 125/256] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:32.772887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 132/257] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:32.777801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d4472-3d1b-43d5-a837-98a89833bf54 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:32.820630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9453bf8-c348-4f22-8db3-71267565ca53 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:32.962288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9453bf8-c348-4f22-8db3-71267565ca53 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:32.992182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:33.003817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78f0fd-6c0a-47c5-a220-87fc2a81de91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:33.004745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78f0fd-6c0a-47c5-a220-87fc2a81de91 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:33.005196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78f0fd-6c0a-47c5-a220-87fc2a81de91 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:33.005677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78f0fd-6c0a-47c5-a220-87fc2a81de91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:33.006342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:33.024033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78f0fd-6c0a-47c5-a220-87fc2a81de91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:33.030670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d4472-3d1b-43d5-a837-98a89833bf54 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-bd78f0fd-6c0a-47c5-a220-87fc2a81de91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:33.031205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:33.032323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:33.033356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:33.035538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 133/258] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:33.042222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78f0fd-6c0a-47c5-a220-87fc2a81de91 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:33.122369 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db28fa-39e8-4922-affe-5bca95123757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:33.123036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db28fa-39e8-4922-affe-5bca95123757 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:33.123317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db28fa-39e8-4922-affe-5bca95123757 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:33.123592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db28fa-39e8-4922-affe-5bca95123757 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:33.154520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db28fa-39e8-4922-affe-5bca95123757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-66db28fa-39e8-4922-affe-5bca95123757 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:33.159839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:33.160947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:33.161928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 126/259] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:33.177622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db28fa-39e8-4922-affe-5bca95123757 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:33.223153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78f0fd-6c0a-47c5-a220-87fc2a81de91 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:33.279289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:33.280180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:33.280630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:33.281043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:33.282017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:33.282468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:33.284083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:33.298064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 134/260] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:33.304996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:33.414527 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:33.447367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db28fa-39e8-4922-affe-5bca95123757 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:33.473880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a511b3-7bac-49ac-9e14-8e1b8ab9ff59 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:33.504460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:33.505077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:33.505350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:33.505618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:33.506303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:33.506673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:33.508063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:33.514466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b947e44-1293-412e-86c5-c3969f4f085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:33.515684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947e44-1293-412e-86c5-c3969f4f085c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:33.516267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947e44-1293-412e-86c5-c3969f4f085c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:33.516766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947e44-1293-412e-86c5-c3969f4f085c None admin] PATH_INFO: `/v3/users/10370723b3eb4c20a1f90fdf745ac0ef` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:33.518190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b947e44-1293-412e-86c5-c3969f4f085c None admin] RBAC: Authorizing `identity:get_user(user_id=10370723b3eb4c20a1f90fdf745ac0ef)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:33.519829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b947e44-1293-412e-86c5-c3969f4f085c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:33.521184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 127/261] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:33.521938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 135/262] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:33 2022] GET /identity/v3/users/10370723b3eb4c20a1f90fdf745ac0ef => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:33.529013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b947e44-1293-412e-86c5-c3969f4f085c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:33.529939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:33.566662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:33.708951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b947e44-1293-412e-86c5-c3969f4f085c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:33.726963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:33.744681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea781fc-8f33-4468-ae77-b3b1046cd524 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:33.749823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c64c0-c811-46f5-9010-930d0ed2eb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:33.750409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c64c0-c811-46f5-9010-930d0ed2eb60 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:33.750639 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c64c0-c811-46f5-9010-930d0ed2eb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:33.750907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c64c0-c811-46f5-9010-930d0ed2eb60 None admin] PATH_INFO: `/v3/users/10370723b3eb4c20a1f90fdf745ac0ef` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:33.756166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c64c0-c811-46f5-9010-930d0ed2eb60 None admin] RBAC: Authorizing `identity:get_user(user_id=10370723b3eb4c20a1f90fdf745ac0ef)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:33.757629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c64c0-c811-46f5-9010-930d0ed2eb60 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:33.761304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 136/263] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:33 2022] GET /identity/v3/users/10370723b3eb4c20a1f90fdf745ac0ef => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:33.766916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c64c0-c811-46f5-9010-930d0ed2eb60 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:33.819401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeac745-985c-41b4-8cfa-336c6b1d367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:33.820013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac745-985c-41b4-8cfa-336c6b1d367a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:33.820291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac745-985c-41b4-8cfa-336c6b1d367a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:33.820590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac745-985c-41b4-8cfa-336c6b1d367a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:33.997848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeac745-985c-41b4-8cfa-336c6b1d367a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:33.999300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeac745-985c-41b4-8cfa-336c6b1d367a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:33.999737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c64c0-c811-46f5-9010-930d0ed2eb60 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:34.043756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418cad9-8e34-434b-94da-f6530bcd7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:34.044382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:34.047434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418cad9-8e34-434b-94da-f6530bcd7e12 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.051226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418cad9-8e34-434b-94da-f6530bcd7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.051752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418cad9-8e34-434b-94da-f6530bcd7e12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.169867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:34.196665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418cad9-8e34-434b-94da-f6530bcd7e12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:34.198846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418cad9-8e34-434b-94da-f6530bcd7e12 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:34.244752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 128/264] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 19:00:34.250587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeac745-985c-41b4-8cfa-336c6b1d367a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:34.352194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 137/265] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:34.363009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418cad9-8e34-434b-94da-f6530bcd7e12 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:34.436858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeac745-985c-41b4-8cfa-336c6b1d367a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:34.490258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:34.510254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956d0a4-6314-418e-af10-02f499d88226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:34.510832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956d0a4-6314-418e-af10-02f499d88226 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.511114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956d0a4-6314-418e-af10-02f499d88226 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.512011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956d0a4-6314-418e-af10-02f499d88226 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.513366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956d0a4-6314-418e-af10-02f499d88226 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:34.515266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956d0a4-6314-418e-af10-02f499d88226 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:34.517717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418cad9-8e34-434b-94da-f6530bcd7e12 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:34.520555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 129/266] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:34 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:34.521030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:34.522239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c0949-d26f-4c3b-8928-25d88547e99f None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.523421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c0949-d26f-4c3b-8928-25d88547e99f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.523904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c0949-d26f-4c3b-8928-25d88547e99f None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.527134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 130/267] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:34.528485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8a484-ebbd-4b89-8cb2-4ca1a8a40420 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.528876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8a484-ebbd-4b89-8cb2-4ca1a8a40420 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.529270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8a484-ebbd-4b89-8cb2-4ca1a8a40420 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.531093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 131/268] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:34.533223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8a484-ebbd-4b89-8cb2-4ca1a8a40420 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:34.563474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cacdfe-8e8d-43a4-9a4d-fd08a8f35904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:34.564105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cacdfe-8e8d-43a4-9a4d-fd08a8f35904 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.564382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cacdfe-8e8d-43a4-9a4d-fd08a8f35904 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.564699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cacdfe-8e8d-43a4-9a4d-fd08a8f35904 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.565915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cacdfe-8e8d-43a4-9a4d-fd08a8f35904 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:34.567479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cacdfe-8e8d-43a4-9a4d-fd08a8f35904 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:34.569420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 138/269] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:34 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:34.570501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee087e3-49c1-4490-b3e6-723e841c1ef4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.570782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee087e3-49c1-4490-b3e6-723e841c1ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.571048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee087e3-49c1-4490-b3e6-723e841c1ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.589845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1ee087e3-49c1-4490-b3e6-723e841c1ef4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:34.591167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1ee087e3-49c1-4490-b3e6-723e841c1ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:34.658510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8a484-ebbd-4b89-8cb2-4ca1a8a40420 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:34.669315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 139/270] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:34.670944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35180df-9af5-4671-aae9-8ef29a96532f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.671430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35180df-9af5-4671-aae9-8ef29a96532f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.671903 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35180df-9af5-4671-aae9-8ef29a96532f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.695750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b35180df-9af5-4671-aae9-8ef29a96532f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:34.697543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b35180df-9af5-4671-aae9-8ef29a96532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:34.723619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:34.724297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.724609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.724881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.725721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=a74bec509592469588b2a87649030293, user.id=10370723b3eb4c20a1f90fdf745ac0ef) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:34.726031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:34.727524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:34.744654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 132/271] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:34 2022] GET /identity/v3/role_assignments?user.id=10370723b3eb4c20a1f90fdf745ac0ef&scope.project.id=a74bec509592469588b2a87649030293&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:34.745747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:34.784291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 140/272] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:34.785566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35180df-9af5-4671-aae9-8ef29a96532f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:34.814520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:34.834491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:34.899261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be72f-246a-4a24-b6d0-887b63c717eb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:34.942125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3377b0-d8f0-42f8-a643-917da6f71a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:34.942782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3377b0-d8f0-42f8-a643-917da6f71a1d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.943023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3377b0-d8f0-42f8-a643-917da6f71a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.943298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3377b0-d8f0-42f8-a643-917da6f71a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.944045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3377b0-d8f0-42f8-a643-917da6f71a1d None admin] RBAC: Adding query filter params (role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=a74bec509592469588b2a87649030293, user.id=10370723b3eb4c20a1f90fdf745ac0ef) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:34.944354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3377b0-d8f0-42f8-a643-917da6f71a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:34.945580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35180df-9af5-4671-aae9-8ef29a96532f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:34.946443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3377b0-d8f0-42f8-a643-917da6f71a1d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:34.958393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 133/273] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:34 2022] GET /identity/v3/role_assignments?user.id=10370723b3eb4c20a1f90fdf745ac0ef&scope.project.id=a74bec509592469588b2a87649030293&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:34.960402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb4fc-b455-4bb2-8588-1a4e8bffd1d3 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:34.960934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb4fc-b455-4bb2-8588-1a4e8bffd1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:34.961400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eb4fc-b455-4bb2-8588-1a4e8bffd1d3 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:34.963077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 134/274] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:34.964078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eb4fc-b455-4bb2-8588-1a4e8bffd1d3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.009958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056212db-7f35-4b66-bac3-55b20f12fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.010619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056212db-7f35-4b66-bac3-55b20f12fff0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.010895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056212db-7f35-4b66-bac3-55b20f12fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.012042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056212db-7f35-4b66-bac3-55b20f12fff0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.025795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056212db-7f35-4b66-bac3-55b20f12fff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-056212db-7f35-4b66-bac3-55b20f12fff0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:35.031159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:35.032482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:35.033536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 141/275] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:35.037905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df076-b726-4805-9801-6ed93b3c41e5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.038122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df076-b726-4805-9801-6ed93b3c41e5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.038382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df076-b726-4805-9801-6ed93b3c41e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.063354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-372df076-b726-4805-9801-6ed93b3c41e5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:35.064607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-372df076-b726-4805-9801-6ed93b3c41e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:35.130205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eb4fc-b455-4bb2-8588-1a4e8bffd1d3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:35.145129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:35.152613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 142/276] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:35.157452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372df076-b726-4805-9801-6ed93b3c41e5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.160989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:35.187584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb32fe-8871-4797-ba35-65702731b764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.188326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb32fe-8871-4797-ba35-65702731b764 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.188692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb32fe-8871-4797-ba35-65702731b764 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.188963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb32fe-8871-4797-ba35-65702731b764 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.190168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb32fe-8871-4797-ba35-65702731b764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:35.191712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb32fe-8871-4797-ba35-65702731b764 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:35.193573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 135/277] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:35.194290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb32fe-8871-4797-ba35-65702731b764 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.284954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372df076-b726-4805-9801-6ed93b3c41e5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:35.319123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb32fe-8871-4797-ba35-65702731b764 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:35.319980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.320623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.320900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.321178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.321837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:35.322140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:35.323579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:35.332651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 143/278] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:35.337302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.352154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb45ec-edb8-4cbc-b9b1-e127d0bafa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.352818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb45ec-edb8-4cbc-b9b1-e127d0bafa78 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.353096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb45ec-edb8-4cbc-b9b1-e127d0bafa78 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.353367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb45ec-edb8-4cbc-b9b1-e127d0bafa78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.364123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb45ec-edb8-4cbc-b9b1-e127d0bafa78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-24eb45ec-edb8-4cbc-b9b1-e127d0bafa78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:35.369214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:35.370080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:35.371669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 136/279] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:35.376931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca64cf-baf3-40cc-a2d5-5b028cc635fa None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.377219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca64cf-baf3-40cc-a2d5-5b028cc635fa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.377500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca64cf-baf3-40cc-a2d5-5b028cc635fa None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.378793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 137/280] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:35.383568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca64cf-baf3-40cc-a2d5-5b028cc635fa None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.399534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:35.427197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:35.439013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94a334-f8f2-4ab3-a5e7-7c511b44eacd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:35.472886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93097442-a94d-42d5-b9ab-25935174382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.473492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93097442-a94d-42d5-b9ab-25935174382b None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.473772 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93097442-a94d-42d5-b9ab-25935174382b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.474046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93097442-a94d-42d5-b9ab-25935174382b None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.474713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93097442-a94d-42d5-b9ab-25935174382b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:35.476154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93097442-a94d-42d5-b9ab-25935174382b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:35.489739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca64cf-baf3-40cc-a2d5-5b028cc635fa None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:35.500035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 144/281] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:00:35 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:35.500873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93097442-a94d-42d5-b9ab-25935174382b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.529488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550d10e-2be8-4387-af82-bb1ecd3cadc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.530141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550d10e-2be8-4387-af82-bb1ecd3cadc7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.530418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550d10e-2be8-4387-af82-bb1ecd3cadc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.530691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550d10e-2be8-4387-af82-bb1ecd3cadc7 None admin] PATH_INFO: `/v3/users/190b069f512e4c608aa58c4de24e4a24` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.550279 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550d10e-2be8-4387-af82-bb1ecd3cadc7 None admin] RBAC: Authorizing `identity:get_user(user_id=190b069f512e4c608aa58c4de24e4a24)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:35.551777 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550d10e-2be8-4387-af82-bb1ecd3cadc7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:35.553663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 138/282] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:35 2022] GET /identity/v3/users/190b069f512e4c608aa58c4de24e4a24 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:35.553663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b25ed-4919-4354-8aac-67d4ba61021f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.553663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b25ed-4919-4354-8aac-67d4ba61021f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.553663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b25ed-4919-4354-8aac-67d4ba61021f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.575898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d14b25ed-4919-4354-8aac-67d4ba61021f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:35.577202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d14b25ed-4919-4354-8aac-67d4ba61021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:35.617920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93097442-a94d-42d5-b9ab-25935174382b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:35.640716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:35.653406 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 139/283] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:35.654502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b25ed-4919-4354-8aac-67d4ba61021f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.671715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.672408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.672702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.672917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.673621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:35.673901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:35.675285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:35.686564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 145/284] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:35.687228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.713440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:35.807925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b25ed-4919-4354-8aac-67d4ba61021f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:35.818424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91e234-ec44-4f78-a62c-08a00e006973 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:35.848376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.849226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.849648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.850189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.851181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] RBAC: Adding query filter params (role.id=46921b1a8b1a4d2d9ad0dd7070581618, user.id=190b069f512e4c608aa58c4de24e4a24, scope.system=all) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:35.851630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:35.853227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:35.856604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78ca84-1eff-4ee1-b1a8-f3ce60104f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:35.857323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78ca84-1eff-4ee1-b1a8-f3ce60104f32 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:35.857683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78ca84-1eff-4ee1-b1a8-f3ce60104f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:35.858029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78ca84-1eff-4ee1-b1a8-f3ce60104f32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:35.860497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 140/285] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Aug 9 19:00:35 2022] GET /identity/v3/role_assignments?user.id=190b069f512e4c608aa58c4de24e4a24&scope.system=all&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:35.861457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.871210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78ca84-1eff-4ee1-b1a8-f3ce60104f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ec78ca84-1eff-4ee1-b1a8-f3ce60104f32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:35.877165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:35.878089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:35.878856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 146/286] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:35.884522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78ca84-1eff-4ee1-b1a8-f3ce60104f32 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:35.972281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:36.020895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:36.033569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16abb-a388-4c3f-800b-473560a6a186 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:36.054335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78ca84-1eff-4ee1-b1a8-f3ce60104f32 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:36.093229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:36.093862 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:36.094152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:36.094424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:36.095111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:36.095415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:36.096878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:36.104737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682f161-2feb-4e19-829b-9892535b1b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:36.105820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682f161-2feb-4e19-829b-9892535b1b70 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:36.106103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682f161-2feb-4e19-829b-9892535b1b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:36.106376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682f161-2feb-4e19-829b-9892535b1b70 None admin] PATH_INFO: `/v3/users/c80bc1629fea4840bcb72d175ac1acf1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:36.112346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682f161-2feb-4e19-829b-9892535b1b70 None admin] RBAC: Authorizing `identity:get_user(user_id=c80bc1629fea4840bcb72d175ac1acf1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:36.113874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682f161-2feb-4e19-829b-9892535b1b70 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:36.115756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 141/287] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:35 2022] GET /identity/v3/users/c80bc1629fea4840bcb72d175ac1acf1 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:36.123229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682f161-2feb-4e19-829b-9892535b1b70 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:36.125761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 147/288] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:36.130794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:36.262188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742c5f3-e30c-4d76-8da4-9b70164f8b92 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:36.266812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:36.275038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682f161-2feb-4e19-829b-9892535b1b70 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:36.329703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f96e-de08-413c-83ee-0be396b7bf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:36.330338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f96e-de08-413c-83ee-0be396b7bf90 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:36.330619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f96e-de08-413c-83ee-0be396b7bf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:36.330889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f96e-de08-413c-83ee-0be396b7bf90 None admin] PATH_INFO: `/v3/users/b7ef258e77e3400c98dc630ac338bdeb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:36.333085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f96e-de08-413c-83ee-0be396b7bf90 None admin] RBAC: Authorizing `identity:get_user(user_id=b7ef258e77e3400c98dc630ac338bdeb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:36.334538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f96e-de08-413c-83ee-0be396b7bf90 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:36.342340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ee8fe-5ed9-4441-864a-0d447fc7055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:36.342982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ee8fe-5ed9-4441-864a-0d447fc7055e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:36.343264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 148/289] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:36 2022] GET /identity/v3/users/b7ef258e77e3400c98dc630ac338bdeb => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:36.343368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ee8fe-5ed9-4441-864a-0d447fc7055e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:36.343633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ee8fe-5ed9-4441-864a-0d447fc7055e None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:36.345933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f96e-de08-413c-83ee-0be396b7bf90 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:36.347628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ee8fe-5ed9-4441-864a-0d447fc7055e None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:36.350205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ee8fe-5ed9-4441-864a-0d447fc7055e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:36.352225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 142/290] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:36 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:36.368591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ee8fe-5ed9-4441-864a-0d447fc7055e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:36.419449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:36.522544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ee8fe-5ed9-4441-864a-0d447fc7055e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:36.565192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb90dd-c645-4820-9ad1-0c77fa3d335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:36.566502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb90dd-c645-4820-9ad1-0c77fa3d335a None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:36.566982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb90dd-c645-4820-9ad1-0c77fa3d335a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:36.567451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb90dd-c645-4820-9ad1-0c77fa3d335a None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293/users/c80bc1629fea4840bcb72d175ac1acf1/roles/585290ba3e014e9e9ae58a7c1d1f1f67` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:36.569824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb90dd-c645-4820-9ad1-0c77fa3d335a None admin] RBAC: Authorizing `identity:create_grant(project_id=a74bec509592469588b2a87649030293, user_id=c80bc1629fea4840bcb72d175ac1acf1, role_id=585290ba3e014e9e9ae58a7c1d1f1f67)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:36.571460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb90dd-c645-4820-9ad1-0c77fa3d335a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:36.582614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f96e-de08-413c-83ee-0be396b7bf90 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:36.597054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 143/291] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:00:36 2022] PUT /identity/v3/projects/a74bec509592469588b2a87649030293/users/c80bc1629fea4840bcb72d175ac1acf1/roles/585290ba3e014e9e9ae58a7c1d1f1f67 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:00:36.630279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2ec7-8476-4f4d-92b0-f8511a07637b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:36.631602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2ec7-8476-4f4d-92b0-f8511a07637b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:36.632124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2ec7-8476-4f4d-92b0-f8511a07637b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:36.632636 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2ec7-8476-4f4d-92b0-f8511a07637b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:36.699758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:36.718684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2ec7-8476-4f4d-92b0-f8511a07637b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:36.720595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2ec7-8476-4f4d-92b0-f8511a07637b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:36.884119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 149/292] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:36.890696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb90dd-c645-4820-9ad1-0c77fa3d335a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:36.990643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:37.005757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb90dd-c645-4820-9ad1-0c77fa3d335a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:37.053191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c013572-159c-4ddd-8883-63b8dbd31c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:37.053837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c013572-159c-4ddd-8883-63b8dbd31c98 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:37.054118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c013572-159c-4ddd-8883-63b8dbd31c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:37.054460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c013572-159c-4ddd-8883-63b8dbd31c98 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:37.055633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c013572-159c-4ddd-8883-63b8dbd31c98 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:37.057128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c013572-159c-4ddd-8883-63b8dbd31c98 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:37.059619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 144/293] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:36 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:37.069545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2ec7-8476-4f4d-92b0-f8511a07637b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:37.189541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:37.226388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2ec7-8476-4f4d-92b0-f8511a07637b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:37.280671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2e22a-c986-4a72-8515-4e60d6d82fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:37.281310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2e22a-c986-4a72-8515-4e60d6d82fea None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:37.281589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2e22a-c986-4a72-8515-4e60d6d82fea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:37.281861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2e22a-c986-4a72-8515-4e60d6d82fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:37.282612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2e22a-c986-4a72-8515-4e60d6d82fea None admin] RBAC: Adding query filter params (role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=a74bec509592469588b2a87649030293, user.id=b7ef258e77e3400c98dc630ac338bdeb) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:37.282920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2e22a-c986-4a72-8515-4e60d6d82fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:37.284376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2e22a-c986-4a72-8515-4e60d6d82fea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:37.293766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 150/294] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:37 2022] GET /identity/v3/role_assignments?user.id=b7ef258e77e3400c98dc630ac338bdeb&scope.project.id=a74bec509592469588b2a87649030293&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:38.060732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead3223-0998-4f01-82e9-6aa68f825037 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.061268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead3223-0998-4f01-82e9-6aa68f825037 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.061268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead3223-0998-4f01-82e9-6aa68f825037 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.063068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 145/295] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:38.068714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf00d-dc65-404b-9d3f-2bda7282e624 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.069157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf00d-dc65-404b-9d3f-2bda7282e624 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.069353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf00d-dc65-404b-9d3f-2bda7282e624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.089906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-19adf00d-dc65-404b-9d3f-2bda7282e624 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:38.091587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-19adf00d-dc65-404b-9d3f-2bda7282e624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:38.162753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 151/296] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:38.186525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead3223-0998-4f01-82e9-6aa68f825037 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:38.247973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2d2ea-6a92-4434-859c-27e61ba8d510 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.248308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2d2ea-6a92-4434-859c-27e61ba8d510 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.248551 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2d2ea-6a92-4434-859c-27e61ba8d510 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.250045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 152/297] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:38.256294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95187edb-5a40-42f3-ac85-92a6caf05d63 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.256879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95187edb-5a40-42f3-ac85-92a6caf05d63 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.257569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95187edb-5a40-42f3-ac85-92a6caf05d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.279180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-95187edb-5a40-42f3-ac85-92a6caf05d63 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:38.280387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-95187edb-5a40-42f3-ac85-92a6caf05d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:38.306669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead3223-0998-4f01-82e9-6aa68f825037 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:38.342747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff2f65-7de0-4309-b3f0-06840aeb290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:38.343302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff2f65-7de0-4309-b3f0-06840aeb290e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.343573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff2f65-7de0-4309-b3f0-06840aeb290e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.343834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff2f65-7de0-4309-b3f0-06840aeb290e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.344952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff2f65-7de0-4309-b3f0-06840aeb290e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:38.345079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:38.346722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff2f65-7de0-4309-b3f0-06840aeb290e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:38.348404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 146/298] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:38.349595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee450b-5eca-4c3c-b251-61f3cef3c4ae None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.349876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee450b-5eca-4c3c-b251-61f3cef3c4ae None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.350147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee450b-5eca-4c3c-b251-61f3cef3c4ae None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.351510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 147/299] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:38.356533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee450b-5eca-4c3c-b251-61f3cef3c4ae None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:38.364731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 153/300] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:38.368459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7e14-340e-4802-9eac-b4af48745466 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.368459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7e14-340e-4802-9eac-b4af48745466 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.368459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7e14-340e-4802-9eac-b4af48745466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.389018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:38.391374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-939d7e14-340e-4802-9eac-b4af48745466 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:38.392715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-939d7e14-340e-4802-9eac-b4af48745466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:38.472834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee450b-5eca-4c3c-b251-61f3cef3c4ae None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:38.474352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 154/301] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:38.480655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7e14-340e-4802-9eac-b4af48745466 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:38.520150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68813644-f6cd-4cad-99c1-49e9f2cfb9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:38.520793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68813644-f6cd-4cad-99c1-49e9f2cfb9fa None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.521071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68813644-f6cd-4cad-99c1-49e9f2cfb9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.521343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68813644-f6cd-4cad-99c1-49e9f2cfb9fa None admin] PATH_INFO: `/v3/groups` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.521965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68813644-f6cd-4cad-99c1-49e9f2cfb9fa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:38.523810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68813644-f6cd-4cad-99c1-49e9f2cfb9fa None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:38.541369 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 148/302] 10.222.0.70 () {60 vars in 1134 bytes} [Tue Aug 9 19:00:38 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:38.541717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68813644-f6cd-4cad-99c1-49e9f2cfb9fa None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:38.629965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7e14-340e-4802-9eac-b4af48745466 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:38.643316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:38.665760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68813644-f6cd-4cad-99c1-49e9f2cfb9fa None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:38.673252 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:38.687378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14e7bb-127f-4d5c-b9f6-c668cac9deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:38.688389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e7bb-127f-4d5c-b9f6-c668cac9deb0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.688924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e7bb-127f-4d5c-b9f6-c668cac9deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.689399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14e7bb-127f-4d5c-b9f6-c668cac9deb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.706971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e869f-563f-463f-b273-8c1d0bc5235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:38.707620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e869f-563f-463f-b273-8c1d0bc5235a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.707896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e869f-563f-463f-b273-8c1d0bc5235a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.708682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e869f-563f-463f-b273-8c1d0bc5235a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.709021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14e7bb-127f-4d5c-b9f6-c668cac9deb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0b14e7bb-127f-4d5c-b9f6-c668cac9deb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:38.716034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:38.717146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:38.718105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 155/303] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:38.722179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14e7bb-127f-4d5c-b9f6-c668cac9deb0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:38.733151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e869f-563f-463f-b273-8c1d0bc5235a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c38e869f-563f-463f-b273-8c1d0bc5235a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:38.738642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:38.739528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:38.740290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 149/304] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:38.745999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e869f-563f-463f-b273-8c1d0bc5235a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:38.857306 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14e7bb-127f-4d5c-b9f6-c668cac9deb0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:38.887210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e869f-563f-463f-b273-8c1d0bc5235a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:38.901386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:38.902393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.903020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.903523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.904368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:38.904887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:38.906369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:38.915156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 156/305] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:38.921056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:38.929120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:38.929960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:38.930978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:38.931435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:38.932292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:38.933894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:38.936651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:38.954327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 150/306] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:38.955058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:38.970047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:38.988159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:39.052028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f0d1b-4829-4550-807e-78b1eda6f6e6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:39.083356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdc9d3-40df-4ab0-93b0-70f3d322918b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:39.090432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb7185-728d-4b05-8d6e-524ae4b259ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:39.091267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb7185-728d-4b05-8d6e-524ae4b259ec None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.091550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb7185-728d-4b05-8d6e-524ae4b259ec None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.091825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb7185-728d-4b05-8d6e-524ae4b259ec None admin] PATH_INFO: `/v3/users/190b069f512e4c608aa58c4de24e4a24` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.092988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb7185-728d-4b05-8d6e-524ae4b259ec None admin] RBAC: Authorizing `identity:get_user(user_id=190b069f512e4c608aa58c4de24e4a24)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:39.094546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb7185-728d-4b05-8d6e-524ae4b259ec None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:39.101007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 157/307] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:38 2022] GET /identity/v3/users/190b069f512e4c608aa58c4de24e4a24 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:39.102228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefb22-02cd-435c-bb7b-5b98481b4344 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.102513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefb22-02cd-435c-bb7b-5b98481b4344 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.102789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefb22-02cd-435c-bb7b-5b98481b4344 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.107929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 158/308] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:39.107929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefb22-02cd-435c-bb7b-5b98481b4344 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:39.124988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f32de0-7dbe-4cfb-9030-706ddd8fe17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:39.125632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f32de0-7dbe-4cfb-9030-706ddd8fe17e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.125916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f32de0-7dbe-4cfb-9030-706ddd8fe17e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.126192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f32de0-7dbe-4cfb-9030-706ddd8fe17e None admin] PATH_INFO: `/v3/users/a00b95b2003f4a48a96412ba05155923` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.141130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f32de0-7dbe-4cfb-9030-706ddd8fe17e None admin] RBAC: Authorizing `identity:get_user(user_id=a00b95b2003f4a48a96412ba05155923)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:39.142524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f32de0-7dbe-4cfb-9030-706ddd8fe17e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:39.144206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 151/309] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:38 2022] GET /identity/v3/users/a00b95b2003f4a48a96412ba05155923 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:39.145426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccc42f-5cfb-4f3e-9653-85d34e4e00ae None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.145708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccc42f-5cfb-4f3e-9653-85d34e4e00ae None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.145979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccc42f-5cfb-4f3e-9653-85d34e4e00ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.165250 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-73ccc42f-5cfb-4f3e-9653-85d34e4e00ae None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:39.166463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-73ccc42f-5cfb-4f3e-9653-85d34e4e00ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:39.231474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefb22-02cd-435c-bb7b-5b98481b4344 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:39.233029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:39.248473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 152/310] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:39.248473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccc42f-5cfb-4f3e-9653-85d34e4e00ae None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:39.249413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:39.271652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee3be2-f379-4ff7-87df-b7811732de06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:39.272336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee3be2-f379-4ff7-87df-b7811732de06 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.272702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee3be2-f379-4ff7-87df-b7811732de06 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.273179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee3be2-f379-4ff7-87df-b7811732de06 None admin] PATH_INFO: `/v3/system/users/190b069f512e4c608aa58c4de24e4a24/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.275338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee3be2-f379-4ff7-87df-b7811732de06 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=190b069f512e4c608aa58c4de24e4a24, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:39.277402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee3be2-f379-4ff7-87df-b7811732de06 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:39.287692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 159/311] 10.222.0.70 () {58 vars in 1334 bytes} [Tue Aug 9 19:00:39 2022] PUT /identity/v3/system/users/190b069f512e4c608aa58c4de24e4a24/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:39.296839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee3be2-f379-4ff7-87df-b7811732de06 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:39.401070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccc42f-5cfb-4f3e-9653-85d34e4e00ae None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:39.447028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee3be2-f379-4ff7-87df-b7811732de06 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:39.448699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:39.449623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.450184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.451289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.451528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] RBAC: Adding query filter params (role.id=585290ba3e014e9e9ae58a7c1d1f1f67, user.id=a00b95b2003f4a48a96412ba05155923, scope.system=all) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:39.451941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:39.454363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:39.460265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 153/312] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Aug 9 19:00:39 2022] GET /identity/v3/role_assignments?user.id=a00b95b2003f4a48a96412ba05155923&scope.system=all&role.id=585290ba3e014e9e9ae58a7c1d1f1f67 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:39.493345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8409aaf-3189-49be-aaac-6e6923ab7fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:39.494140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8409aaf-3189-49be-aaac-6e6923ab7fec None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.494489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8409aaf-3189-49be-aaac-6e6923ab7fec None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.494826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8409aaf-3189-49be-aaac-6e6923ab7fec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.507161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8409aaf-3189-49be-aaac-6e6923ab7fec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a8409aaf-3189-49be-aaac-6e6923ab7fec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:39.512717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:39.513674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:39.514448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 160/313] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:39.519994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:39.588044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:39.639125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdfdb9-a96a-4e81-adf1-85bab3afd4cf None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:39.677503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:39.678385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.678837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.679276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.680087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:39.680558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:39.682064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:39.692327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 154/314] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:39.697451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8409aaf-3189-49be-aaac-6e6923ab7fec None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:39.743121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:39.811086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8409aaf-3189-49be-aaac-6e6923ab7fec None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:39.849030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a5ac5-23a2-479b-b79b-13198906f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:39.849666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a5ac5-23a2-479b-b79b-13198906f1cc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:39.849946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a5ac5-23a2-479b-b79b-13198906f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:39.850222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a5ac5-23a2-479b-b79b-13198906f1cc None admin] PATH_INFO: `/v3/users/c80bc1629fea4840bcb72d175ac1acf1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:39.851339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a5ac5-23a2-479b-b79b-13198906f1cc None admin] RBAC: Authorizing `identity:get_user(user_id=c80bc1629fea4840bcb72d175ac1acf1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:39.852826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a5ac5-23a2-479b-b79b-13198906f1cc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:39.854603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 161/315] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:39 2022] GET /identity/v3/users/c80bc1629fea4840bcb72d175ac1acf1 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:39.860735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:39.974762 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6cc5e-5518-4ebc-892d-7a84b394f7b4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:39.996610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:40.012852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74de32-e263-4ac4-b223-dc6ebfb04929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:40.013461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74de32-e263-4ac4-b223-dc6ebfb04929 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:40.013739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74de32-e263-4ac4-b223-dc6ebfb04929 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:40.014008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74de32-e263-4ac4-b223-dc6ebfb04929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:40.087964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74de32-e263-4ac4-b223-dc6ebfb04929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:40.089512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74de32-e263-4ac4-b223-dc6ebfb04929 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:40.194995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 155/316] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:40.201963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a5ac5-23a2-479b-b79b-13198906f1cc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:40.309254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a5ac5-23a2-479b-b79b-13198906f1cc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:40.318456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:40.344787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5e6f5-3f5b-4c61-af79-298642ac26bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:40.345415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5e6f5-3f5b-4c61-af79-298642ac26bc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:40.345692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5e6f5-3f5b-4c61-af79-298642ac26bc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:40.345963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5e6f5-3f5b-4c61-af79-298642ac26bc None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:40.347139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5e6f5-3f5b-4c61-af79-298642ac26bc None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:40.348652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5e6f5-3f5b-4c61-af79-298642ac26bc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:40.350569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 162/317] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:40 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:40.357038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74de32-e263-4ac4-b223-dc6ebfb04929 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:40.380286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:40.471268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74de32-e263-4ac4-b223-dc6ebfb04929 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:40.513240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e261-f546-449c-bad0-e40e66190a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:40.514110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e261-f546-449c-bad0-e40e66190a33 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:40.514532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e261-f546-449c-bad0-e40e66190a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:40.514974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e261-f546-449c-bad0-e40e66190a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:40.515917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720e261-f546-449c-bad0-e40e66190a33 None admin] RBAC: Adding query filter params (role.id=585290ba3e014e9e9ae58a7c1d1f1f67, scope.project.id=a74bec509592469588b2a87649030293, user.id=c80bc1629fea4840bcb72d175ac1acf1) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:40.516414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720e261-f546-449c-bad0-e40e66190a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:40.518035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720e261-f546-449c-bad0-e40e66190a33 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:40.524978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 156/318] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:00:40 2022] GET /identity/v3/role_assignments?user.id=c80bc1629fea4840bcb72d175ac1acf1&scope.project.id=a74bec509592469588b2a87649030293&role.id=585290ba3e014e9e9ae58a7c1d1f1f67 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:40.701396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fc38b-fc3c-4450-bfe5-77d84e93e704 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:40.701767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fc38b-fc3c-4450-bfe5-77d84e93e704 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:40.701961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fc38b-fc3c-4450-bfe5-77d84e93e704 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:40.703423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 163/319] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:40.708983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfd657-df71-4d13-be6c-2ef5150ba959 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:40.709266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfd657-df71-4d13-be6c-2ef5150ba959 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:40.709536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfd657-df71-4d13-be6c-2ef5150ba959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:40.728780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-08cfd657-df71-4d13-be6c-2ef5150ba959 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:40.730159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-08cfd657-df71-4d13-be6c-2ef5150ba959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:40.805989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 157/320] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:40.828851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fc38b-fc3c-4450-bfe5-77d84e93e704 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:40.932365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fc38b-fc3c-4450-bfe5-77d84e93e704 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:40.968067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38955c9d-bd17-4127-91c1-3f7bd405feb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:40.968646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38955c9d-bd17-4127-91c1-3f7bd405feb6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:40.968921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38955c9d-bd17-4127-91c1-3f7bd405feb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:40.969194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38955c9d-bd17-4127-91c1-3f7bd405feb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:40.970282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38955c9d-bd17-4127-91c1-3f7bd405feb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:40.971725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38955c9d-bd17-4127-91c1-3f7bd405feb6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:40.973463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 164/321] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:40.979261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfd657-df71-4d13-be6c-2ef5150ba959 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:40.994705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:41.083881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfd657-df71-4d13-be6c-2ef5150ba959 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:41.118066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c86f1b-56fa-4814-865d-aab17a9ddd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:41.118693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c86f1b-56fa-4814-865d-aab17a9ddd8e None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:41.118935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c86f1b-56fa-4814-865d-aab17a9ddd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:41.119210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c86f1b-56fa-4814-865d-aab17a9ddd8e None admin] PATH_INFO: `/v3/groups` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:41.119843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c86f1b-56fa-4814-865d-aab17a9ddd8e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:41.121885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c86f1b-56fa-4814-865d-aab17a9ddd8e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:41.139580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 158/322] 10.222.0.70 () {60 vars in 1134 bytes} [Tue Aug 9 19:00:40 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:00:41.429544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad4698-cfa4-42f4-b1ae-1329916e3d66 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:41.430063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad4698-cfa4-42f4-b1ae-1329916e3d66 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:41.430063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad4698-cfa4-42f4-b1ae-1329916e3d66 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:41.431560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 165/323] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:41.437393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d831-1de6-46c2-9c8f-e4991b1f73ae None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:41.437678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d831-1de6-46c2-9c8f-e4991b1f73ae None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:41.437953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51d831-1de6-46c2-9c8f-e4991b1f73ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:41.461143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0d51d831-1de6-46c2-9c8f-e4991b1f73ae None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:41.462348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0d51d831-1de6-46c2-9c8f-e4991b1f73ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:41.536557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 159/324] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:41.558805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad4698-cfa4-42f4-b1ae-1329916e3d66 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:41.608736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a66ff-edad-467a-a147-afaf0a24e06e None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:41.609056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a66ff-edad-467a-a147-afaf0a24e06e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:41.609283 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a66ff-edad-467a-a147-afaf0a24e06e None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:41.610745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 160/325] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:41.613957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:41.616336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a54c10-22e5-4f6f-92d0-a7632c3bff1a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:41.616845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a54c10-22e5-4f6f-92d0-a7632c3bff1a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:41.616937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a54c10-22e5-4f6f-92d0-a7632c3bff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:41.626342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:41.640003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-04a54c10-22e5-4f6f-92d0-a7632c3bff1a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:41.641458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-04a54c10-22e5-4f6f-92d0-a7632c3bff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:41.684161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad4698-cfa4-42f4-b1ae-1329916e3d66 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:41.724570 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 161/326] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:41.726663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0672-d5d2-4baf-88b6-b289fae7694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:41.727300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0672-d5d2-4baf-88b6-b289fae7694b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:41.727577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0672-d5d2-4baf-88b6-b289fae7694b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:41.727843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0672-d5d2-4baf-88b6-b289fae7694b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:41.741215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c0672-d5d2-4baf-88b6-b289fae7694b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1c1c0672-d5d2-4baf-88b6-b289fae7694b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:41.746655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:41.747678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:41.748464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 166/327] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:41.753864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0672-d5d2-4baf-88b6-b289fae7694b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:41.753991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a54c10-22e5-4f6f-92d0-a7632c3bff1a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:41.907478 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0672-d5d2-4baf-88b6-b289fae7694b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:41.907794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a54c10-22e5-4f6f-92d0-a7632c3bff1a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:41.909968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:41.947063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b2669-1d9c-4f1d-9b00-f913780b550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:41.947769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b2669-1d9c-4f1d-9b00-f913780b550a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:41.948163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b2669-1d9c-4f1d-9b00-f913780b550a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:41.948482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b2669-1d9c-4f1d-9b00-f913780b550a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:41.951676 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:41.963838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b2669-1d9c-4f1d-9b00-f913780b550a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:41.965523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:41.966145 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:41.966421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:41.966691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:41.967423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:41.967734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:41.969158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-982b2669-1d9c-4f1d-9b00-f913780b550a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:41.969411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:41.970256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:41.971034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 162/328] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:41.976261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b2669-1d9c-4f1d-9b00-f913780b550a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:41.984051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 167/329] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:41.989429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:42.126750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be60d70-2c17-472a-bc72-6330a3b40804 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:42.137305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b2669-1d9c-4f1d-9b00-f913780b550a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:42.162392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ccce3-7617-4909-a708-1ce1657ae8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:42.162952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ccce3-7617-4909-a708-1ce1657ae8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:42.163234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ccce3-7617-4909-a708-1ce1657ae8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:42.163499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ccce3-7617-4909-a708-1ce1657ae8a5 None admin] PATH_INFO: `/v3/users/190b069f512e4c608aa58c4de24e4a24` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:42.164711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ccce3-7617-4909-a708-1ce1657ae8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=190b069f512e4c608aa58c4de24e4a24)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:42.166160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ccce3-7617-4909-a708-1ce1657ae8a5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:42.172838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 168/330] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:41 2022] GET /identity/v3/users/190b069f512e4c608aa58c4de24e4a24 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:42.175467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ccce3-7617-4909-a708-1ce1657ae8a5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:42.211739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:42.219754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:42.220466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:42.220755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:42.221030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:42.221709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:42.222010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:42.223391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:42.247677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 163/331] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:42.253471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:42.281971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ccce3-7617-4909-a708-1ce1657ae8a5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:42.317307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:42.317921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:42.318195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:42.318467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:42.319194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] RBAC: Adding query filter params (role.id=46921b1a8b1a4d2d9ad0dd7070581618, user.id=190b069f512e4c608aa58c4de24e4a24, scope.system=all) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:42.319494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:42.320947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:42.340629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 169/332] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Aug 9 19:00:42 2022] GET /identity/v3/role_assignments?user.id=190b069f512e4c608aa58c4de24e4a24&scope.system=all&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:42.363490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:42.413767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0fc60-2fb7-4798-ae28-4918de7c173f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:42.457255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366700c-a8c9-4349-b1b9-63e289cea9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:42.457912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366700c-a8c9-4349-b1b9-63e289cea9da None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:42.458191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366700c-a8c9-4349-b1b9-63e289cea9da None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:42.458476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366700c-a8c9-4349-b1b9-63e289cea9da None admin] PATH_INFO: `/v3/users/a00b95b2003f4a48a96412ba05155923` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:42.465890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366700c-a8c9-4349-b1b9-63e289cea9da None admin] RBAC: Authorizing `identity:get_user(user_id=a00b95b2003f4a48a96412ba05155923)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:42.467393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366700c-a8c9-4349-b1b9-63e289cea9da None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:42.469573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 164/333] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:42 2022] GET /identity/v3/users/a00b95b2003f4a48a96412ba05155923 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:42.476491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:42.598937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:42.607726 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898f1d5-d5bf-4259-9287-cfc99d1fd4b3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:42.648461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97f260-0ba0-4579-956c-044b07c1a200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:42.649122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97f260-0ba0-4579-956c-044b07c1a200 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:42.649397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97f260-0ba0-4579-956c-044b07c1a200 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:42.649667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97f260-0ba0-4579-956c-044b07c1a200 None admin] PATH_INFO: `/v3/system/users/a00b95b2003f4a48a96412ba05155923/roles/585290ba3e014e9e9ae58a7c1d1f1f67` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:42.651062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97f260-0ba0-4579-956c-044b07c1a200 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a00b95b2003f4a48a96412ba05155923, role_id=585290ba3e014e9e9ae58a7c1d1f1f67)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:42.652557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97f260-0ba0-4579-956c-044b07c1a200 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:42.666429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 170/334] 10.222.0.70 () {58 vars in 1334 bytes} [Tue Aug 9 19:00:42 2022] PUT /identity/v3/system/users/a00b95b2003f4a48a96412ba05155923/roles/585290ba3e014e9e9ae58a7c1d1f1f67 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:43.794430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c93f4-b8e0-4184-8073-d29dbee6cdd2 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:43.794931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c93f4-b8e0-4184-8073-d29dbee6cdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:43.794931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c93f4-b8e0-4184-8073-d29dbee6cdd2 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:43.796598 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 165/335] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:43.802034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c70e8-9a47-48d5-8648-3cf77578db4b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:43.802472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c70e8-9a47-48d5-8648-3cf77578db4b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:43.802656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c70e8-9a47-48d5-8648-3cf77578db4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:43.826442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7b0c70e8-9a47-48d5-8648-3cf77578db4b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:43.827641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7b0c70e8-9a47-48d5-8648-3cf77578db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:43.908266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddab0a-6612-4e23-af9f-ec66a7bbd9ac None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:43.908948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddab0a-6612-4e23-af9f-ec66a7bbd9ac None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:43.909446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddab0a-6612-4e23-af9f-ec66a7bbd9ac None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:43.911343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 166/336] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:43.915302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 171/337] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:43.927615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e558-4603-4d5f-8335-bcba9109b45b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:43.927943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e558-4603-4d5f-8335-bcba9109b45b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:43.928168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e558-4603-4d5f-8335-bcba9109b45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:43.943592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c70e8-9a47-48d5-8648-3cf77578db4b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:43.955071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9f63e558-4603-4d5f-8335-bcba9109b45b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:43.956672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9f63e558-4603-4d5f-8335-bcba9109b45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:44.056854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:44.080563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 167/338] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:44.088786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c70e8-9a47-48d5-8648-3cf77578db4b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:44.127386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63e558-4603-4d5f-8335-bcba9109b45b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:44.155759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e6d2d-0279-4fd4-80b7-25e1b5491ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:44.156404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e6d2d-0279-4fd4-80b7-25e1b5491ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:44.156706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e6d2d-0279-4fd4-80b7-25e1b5491ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:44.156976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e6d2d-0279-4fd4-80b7-25e1b5491ee1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:44.185519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e6d2d-0279-4fd4-80b7-25e1b5491ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-326e6d2d-0279-4fd4-80b7-25e1b5491ee1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:44.200857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:44.201758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:44.202533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 172/339] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:44.209621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e6d2d-0279-4fd4-80b7-25e1b5491ee1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:44.245210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:44.350711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63e558-4603-4d5f-8335-bcba9109b45b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:44.395140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e6d2d-0279-4fd4-80b7-25e1b5491ee1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:44.459801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495db80-1e74-418d-986c-35bd9bff29a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:44.460474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495db80-1e74-418d-986c-35bd9bff29a5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:44.460756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495db80-1e74-418d-986c-35bd9bff29a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:44.461168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495db80-1e74-418d-986c-35bd9bff29a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:44.462490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:44.463296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:44.463590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:44.463864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:44.465484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:44.465933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:44.467413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:44.483214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 173/340] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:44.489049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:44.490020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495db80-1e74-418d-986c-35bd9bff29a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-5495db80-1e74-418d-986c-35bd9bff29a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:44.494799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:44.496577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:44.497386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 168/341] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:44.504556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495db80-1e74-418d-986c-35bd9bff29a5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:44.511761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:44.654916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b8f62-570a-44aa-9a4b-8ffb5728a62e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:44.705202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5594e-fe68-447f-b889-1280048c97dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:44.705834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5594e-fe68-447f-b889-1280048c97dd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:44.706117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5594e-fe68-447f-b889-1280048c97dd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:44.706390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5594e-fe68-447f-b889-1280048c97dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:44.734016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495db80-1e74-418d-986c-35bd9bff29a5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:44.778413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:44.818802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5594e-fe68-447f-b889-1280048c97dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:44.820360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5594e-fe68-447f-b889-1280048c97dd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:44.836841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:44.869154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:44.869797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:44.870046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:44.870395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:44.871081 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:44.871383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:44.872797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:44.891232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 169/342] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:44.897045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:44.954647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 174/343] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 19:00:44.970748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5594e-fe68-447f-b889-1280048c97dd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:45.032544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a2d66-f02d-408d-884c-beec57db7adf None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:45.070639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e668e-5108-4f53-a407-46c223722cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:45.071255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e668e-5108-4f53-a407-46c223722cff None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.071530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e668e-5108-4f53-a407-46c223722cff None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.071800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e668e-5108-4f53-a407-46c223722cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.114000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5594e-fe68-447f-b889-1280048c97dd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:45.130559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:45.147107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:45.165275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e668e-5108-4f53-a407-46c223722cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:45.166686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e668e-5108-4f53-a407-46c223722cff None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:45.181127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473be8-3b7a-4b7f-945f-f29dcb74e85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:45.182011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473be8-3b7a-4b7f-945f-f29dcb74e85f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.182436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473be8-3b7a-4b7f-945f-f29dcb74e85f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.183480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473be8-3b7a-4b7f-945f-f29dcb74e85f None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.185767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe473be8-3b7a-4b7f-945f-f29dcb74e85f None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:45.187706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe473be8-3b7a-4b7f-945f-f29dcb74e85f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:45.189861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 175/344] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:44 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:45.195471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473be8-3b7a-4b7f-945f-f29dcb74e85f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:45.304774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 170/345] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:45.306317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac2e0a-2fa9-44fb-a76a-d1d06ab3f17d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.306317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac2e0a-2fa9-44fb-a76a-d1d06ab3f17d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.306533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac2e0a-2fa9-44fb-a76a-d1d06ab3f17d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.310464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 171/346] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:45.310464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac2e0a-2fa9-44fb-a76a-d1d06ab3f17d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:45.356304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe473be8-3b7a-4b7f-945f-f29dcb74e85f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:45.406213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c3c7e-2f95-4c2b-9f8b-13d0e259ca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:45.407217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c3c7e-2f95-4c2b-9f8b-13d0e259ca48 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.407759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c3c7e-2f95-4c2b-9f8b-13d0e259ca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.408236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c3c7e-2f95-4c2b-9f8b-13d0e259ca48 None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.418444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c3c7e-2f95-4c2b-9f8b-13d0e259ca48 None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:45.420398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c3c7e-2f95-4c2b-9f8b-13d0e259ca48 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:45.423753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 176/347] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:45 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:45.425282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750f453-ddf1-4df4-a031-295e22dd300d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.425770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750f453-ddf1-4df4-a031-295e22dd300d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.426258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750f453-ddf1-4df4-a031-295e22dd300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.450738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7750f453-ddf1-4df4-a031-295e22dd300d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:45.452318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7750f453-ddf1-4df4-a031-295e22dd300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:45.455493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:45.489137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac2e0a-2fa9-44fb-a76a-d1d06ab3f17d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:45.491058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:45.529534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 177/348] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:45.530426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750f453-ddf1-4df4-a031-295e22dd300d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:45.557268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4619870-6f99-4bb7-a9c0-52adb3863195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:45.558530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4619870-6f99-4bb7-a9c0-52adb3863195 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.558937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4619870-6f99-4bb7-a9c0-52adb3863195 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.559267 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4619870-6f99-4bb7-a9c0-52adb3863195 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.564405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4619870-6f99-4bb7-a9c0-52adb3863195 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:45.565883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4619870-6f99-4bb7-a9c0-52adb3863195 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:45.568486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 172/349] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:45 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:45.569239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4619870-6f99-4bb7-a9c0-52adb3863195 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:45.648257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750f453-ddf1-4df4-a031-295e22dd300d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:45.687911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:45.688563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.688844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.689118 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.689855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] RBAC: Adding query filter params (group.id=c4c39efb086f4a0f9f6a2386ea13ab16, role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=57d897223df84e6ba0e4445299c6559e) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:45.690168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:45.691609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:45.697776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 178/350] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:45 2022] GET /identity/v3/role_assignments?group.id=c4c39efb086f4a0f9f6a2386ea13ab16&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:45.699264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:45.749542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:45.782716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4619870-6f99-4bb7-a9c0-52adb3863195 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:45.832451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8a082-3546-4c63-9453-9f03bdf48fb8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:45.873847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a7d61-bad2-419d-8263-3ef2668f5fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:45.874504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7d61-bad2-419d-8263-3ef2668f5fbf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.874785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7d61-bad2-419d-8263-3ef2668f5fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.875056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7d61-bad2-419d-8263-3ef2668f5fbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.900645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a7d61-bad2-419d-8263-3ef2668f5fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c25a7d61-bad2-419d-8263-3ef2668f5fbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:45.913212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:45.914197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:45.915005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:45.915613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 173/351] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:45.916800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662fd16-9532-4f0b-9f9f-db04db374441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:45.918189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662fd16-9532-4f0b-9f9f-db04db374441 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:45.918677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662fd16-9532-4f0b-9f9f-db04db374441 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:45.919008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662fd16-9532-4f0b-9f9f-db04db374441 None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:45.920513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662fd16-9532-4f0b-9f9f-db04db374441 None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:45.921817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662fd16-9532-4f0b-9f9f-db04db374441 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:45.922017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a7d61-bad2-419d-8263-3ef2668f5fbf None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:45.923544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 179/352] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:45 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:45.928108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:45.935875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662fd16-9532-4f0b-9f9f-db04db374441 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:46.051610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a7d61-bad2-419d-8263-3ef2668f5fbf None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:46.096809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd64d6-d614-43b3-8a93-20c953fd8cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:46.097348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd64d6-d614-43b3-8a93-20c953fd8cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.097623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd64d6-d614-43b3-8a93-20c953fd8cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.097891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd64d6-d614-43b3-8a93-20c953fd8cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.098561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd64d6-d614-43b3-8a93-20c953fd8cf3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:46.098924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd64d6-d614-43b3-8a93-20c953fd8cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:46.100354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd64d6-d614-43b3-8a93-20c953fd8cf3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:46.114634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 174/353] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:46.116045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136b373-485e-40f8-9984-ff94291ae406 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.116329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136b373-485e-40f8-9984-ff94291ae406 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.116696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136b373-485e-40f8-9984-ff94291ae406 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.117975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 175/354] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:46.119269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d60598-0db6-4cef-b71b-e31796deccbb None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.119540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d60598-0db6-4cef-b71b-e31796deccbb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.119866 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d60598-0db6-4cef-b71b-e31796deccbb None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.119971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662fd16-9532-4f0b-9f9f-db04db374441 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:46.125720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 176/355] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:46.126664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d60598-0db6-4cef-b71b-e31796deccbb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:46.130303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:46.161988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71568ae-c55c-4961-8b5f-75bb84524e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:46.162666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71568ae-c55c-4961-8b5f-75bb84524e2f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.162945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71568ae-c55c-4961-8b5f-75bb84524e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.163239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71568ae-c55c-4961-8b5f-75bb84524e2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.163961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71568ae-c55c-4961-8b5f-75bb84524e2f None admin] RBAC: Adding query filter params (group.id=c4c39efb086f4a0f9f6a2386ea13ab16, role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=57d897223df84e6ba0e4445299c6559e) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:46.164246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71568ae-c55c-4961-8b5f-75bb84524e2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:46.165707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71568ae-c55c-4961-8b5f-75bb84524e2f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:46.172613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 180/356] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:45 2022] GET /identity/v3/role_assignments?group.id=c4c39efb086f4a0f9f6a2386ea13ab16&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:46.173837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3c113-de68-4d47-b1fe-08e73a94d740 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.174119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3c113-de68-4d47-b1fe-08e73a94d740 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.174393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3c113-de68-4d47-b1fe-08e73a94d740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.194927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fca3c113-de68-4d47-b1fe-08e73a94d740 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:46.196136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fca3c113-de68-4d47-b1fe-08e73a94d740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:46.209404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:46.253841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d60598-0db6-4cef-b71b-e31796deccbb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:46.278706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 181/357] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:46.280252 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186233bc-40ba-4f10-813f-fab476988b08 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.280738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186233bc-40ba-4f10-813f-fab476988b08 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.281212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186233bc-40ba-4f10-813f-fab476988b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.295242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba923c1-b7a5-4515-8bd4-274719520c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:46.296063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba923c1-b7a5-4515-8bd4-274719520c30 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.296408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba923c1-b7a5-4515-8bd4-274719520c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.296768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba923c1-b7a5-4515-8bd4-274719520c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.302133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-186233bc-40ba-4f10-813f-fab476988b08 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:46.303645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-186233bc-40ba-4f10-813f-fab476988b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:46.378848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 182/358] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:46.379973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186233bc-40ba-4f10-813f-fab476988b08 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:46.411784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba923c1-b7a5-4515-8bd4-274719520c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:46.413425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba923c1-b7a5-4515-8bd4-274719520c30 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:46.422733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:46.504402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186233bc-40ba-4f10-813f-fab476988b08 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:46.528954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:46.548081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750151e-2535-491c-b3ef-63c72648953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:46.548896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750151e-2535-491c-b3ef-63c72648953e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.549298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750151e-2535-491c-b3ef-63c72648953e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.549664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750151e-2535-491c-b3ef-63c72648953e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.562914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750151e-2535-491c-b3ef-63c72648953e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5750151e-2535-491c-b3ef-63c72648953e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:46.568020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:46.568941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:46.569886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 183/359] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:46.571195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750151e-2535-491c-b3ef-63c72648953e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:46.576275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 177/360] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:46.578199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba923c1-b7a5-4515-8bd4-274719520c30 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:46.663241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:46.680713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750151e-2535-491c-b3ef-63c72648953e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:46.689261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba923c1-b7a5-4515-8bd4-274719520c30 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:46.720960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48effbfa-7695-4d1b-93ff-0c983160b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:46.721869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48effbfa-7695-4d1b-93ff-0c983160b16b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.722396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48effbfa-7695-4d1b-93ff-0c983160b16b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.722810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48effbfa-7695-4d1b-93ff-0c983160b16b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.728433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef68044-851f-441a-ac95-3bc628b02c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:46.729188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef68044-851f-441a-ac95-3bc628b02c9a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.729610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef68044-851f-441a-ac95-3bc628b02c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.730480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef68044-851f-441a-ac95-3bc628b02c9a None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.731268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef68044-851f-441a-ac95-3bc628b02c9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:46.731879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef68044-851f-441a-ac95-3bc628b02c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:46.733464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef68044-851f-441a-ac95-3bc628b02c9a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:46.735567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48effbfa-7695-4d1b-93ff-0c983160b16b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-48effbfa-7695-4d1b-93ff-0c983160b16b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:46.740540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:46.741445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:46.742237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:46.744598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 184/362] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:46.746203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48effbfa-7695-4d1b-93ff-0c983160b16b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:46.748543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 178/362] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:46.749917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d6626-a401-4e1c-b3ad-adcc5774ef25 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.750234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d6626-a401-4e1c-b3ad-adcc5774ef25 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.750682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d6626-a401-4e1c-b3ad-adcc5774ef25 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.752541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 179/363] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:46.752763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d6626-a401-4e1c-b3ad-adcc5774ef25 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:46.817857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:46.860884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48effbfa-7695-4d1b-93ff-0c983160b16b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:46.873148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d6626-a401-4e1c-b3ad-adcc5774ef25 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:46.898154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b49810-8ac4-4ab1-b6ab-edb522fcf648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:46.898785 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49810-8ac4-4ab1-b6ab-edb522fcf648 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.899065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49810-8ac4-4ab1-b6ab-edb522fcf648 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.899330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49810-8ac4-4ab1-b6ab-edb522fcf648 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.900464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b49810-8ac4-4ab1-b6ab-edb522fcf648 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:46.902228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b49810-8ac4-4ab1-b6ab-edb522fcf648 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:46.903869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 185/364] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:46 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:46.904713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b49810-8ac4-4ab1-b6ab-edb522fcf648 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:46.911776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2348e3d-7286-4116-8343-2970f204150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:46.912389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348e3d-7286-4116-8343-2970f204150e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.912701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348e3d-7286-4116-8343-2970f204150e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.912978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2348e3d-7286-4116-8343-2970f204150e None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.913805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2348e3d-7286-4116-8343-2970f204150e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:46.914061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2348e3d-7286-4116-8343-2970f204150e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:46.915439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2348e3d-7286-4116-8343-2970f204150e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:46.921110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:46.928755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 180/365] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:46.930023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3703356-21c2-42c3-8ac8-984a261a4392 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:46.930320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3703356-21c2-42c3-8ac8-984a261a4392 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:46.930597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3703356-21c2-42c3-8ac8-984a261a4392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:46.958084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b3703356-21c2-42c3-8ac8-984a261a4392 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:46.959338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b3703356-21c2-42c3-8ac8-984a261a4392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:47.035035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b49810-8ac4-4ab1-b6ab-edb522fcf648 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:47.060920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 181/366] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:47.062235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3703356-21c2-42c3-8ac8-984a261a4392 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:47.088287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e438c0-ecea-4f25-8035-6200485660e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:47.089482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e438c0-ecea-4f25-8035-6200485660e0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:47.089766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e438c0-ecea-4f25-8035-6200485660e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:47.090037 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e438c0-ecea-4f25-8035-6200485660e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:47.097902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:47.187024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e438c0-ecea-4f25-8035-6200485660e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:47.189091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e438c0-ecea-4f25-8035-6200485660e0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:47.189516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3703356-21c2-42c3-8ac8-984a261a4392 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:47.209155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:47.247559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c138c7-1cc4-4071-b9d5-f1e74cee6b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:47.249251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c138c7-1cc4-4071-b9d5-f1e74cee6b6c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:47.249729 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c138c7-1cc4-4071-b9d5-f1e74cee6b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:47.250196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c138c7-1cc4-4071-b9d5-f1e74cee6b6c None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:47.252500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c138c7-1cc4-4071-b9d5-f1e74cee6b6c None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:47.254743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c138c7-1cc4-4071-b9d5-f1e74cee6b6c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:47.257249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 182/367] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:47 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:47.258492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c138c7-1cc4-4071-b9d5-f1e74cee6b6c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:47.337261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 186/368] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:47.338581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e438c0-ecea-4f25-8035-6200485660e0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:47.367639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:47.382622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c138c7-1cc4-4071-b9d5-f1e74cee6b6c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:47.426744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8548-5c43-41ed-9686-ad73daf7e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:47.427362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8548-5c43-41ed-9686-ad73daf7e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:47.427641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8548-5c43-41ed-9686-ad73daf7e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:47.427914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8548-5c43-41ed-9686-ad73daf7e9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:47.471475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e438c0-ecea-4f25-8035-6200485660e0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:47.491563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:47.511571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c8548-5c43-41ed-9686-ad73daf7e9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:47.513064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c8548-5c43-41ed-9686-ad73daf7e9a1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:47.519911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40484760-77d0-46e6-a321-2ba9da968ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:47.520582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40484760-77d0-46e6-a321-2ba9da968ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:47.520861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40484760-77d0-46e6-a321-2ba9da968ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:47.521126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40484760-77d0-46e6-a321-2ba9da968ba2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:47.535018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40484760-77d0-46e6-a321-2ba9da968ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-40484760-77d0-46e6-a321-2ba9da968ba2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:47.542791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:47.543774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:47.544573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 187/369] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:47.548119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40484760-77d0-46e6-a321-2ba9da968ba2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:47.624246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:47.640922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 183/370] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:47.646845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8548-5c43-41ed-9686-ad73daf7e9a1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:47.690813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40484760-77d0-46e6-a321-2ba9da968ba2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:47.738673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:47.739199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:47.739509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:47.739747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:47.740493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] RBAC: Adding query filter params (group.id=c4c39efb086f4a0f9f6a2386ea13ab16, role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=a74bec509592469588b2a87649030293) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:47.740800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:47.742781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:47.752140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 188/371] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:47 2022] GET /identity/v3/role_assignments?group.id=c4c39efb086f4a0f9f6a2386ea13ab16&scope.project.id=a74bec509592469588b2a87649030293&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:47.753973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:47.802391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8548-5c43-41ed-9686-ad73daf7e9a1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:47.855283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:47.874868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9df625-62e1-4840-a1dd-4c057e3fa8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:47.875636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9df625-62e1-4840-a1dd-4c057e3fa8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:47.875915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9df625-62e1-4840-a1dd-4c057e3fa8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:47.876193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9df625-62e1-4840-a1dd-4c057e3fa8b1 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:47.877567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9df625-62e1-4840-a1dd-4c057e3fa8b1 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:47.879015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9df625-62e1-4840-a1dd-4c057e3fa8b1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:47.881032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 184/372] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:47 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:47.881906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9df625-62e1-4840-a1dd-4c057e3fa8b1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:47.998381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40596a6-856d-46b9-9ffe-b4447c7a5fc6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:48.005526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:48.065194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:48.066622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.066901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.067170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.067840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:48.068135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:48.069552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:48.087150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 189/373] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:48.090343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:48.101268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9df625-62e1-4840-a1dd-4c057e3fa8b1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:48.183279 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101ee64-4540-4846-bc35-e2602046ba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:48.183884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ee64-4540-4846-bc35-e2602046ba54 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.184160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ee64-4540-4846-bc35-e2602046ba54 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.184428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ee64-4540-4846-bc35-e2602046ba54 None admin] PATH_INFO: `/v3/projects/de422168330743c9990b96b3dab6b6f7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.188506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101ee64-4540-4846-bc35-e2602046ba54 None admin] RBAC: Authorizing `identity:get_project(project_id=de422168330743c9990b96b3dab6b6f7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:48.189950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101ee64-4540-4846-bc35-e2602046ba54 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:48.191594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 185/374] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:47 2022] GET /identity/v3/projects/de422168330743c9990b96b3dab6b6f7 => generated 344 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:48.199526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101ee64-4540-4846-bc35-e2602046ba54 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:48.271036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:48.275651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e8bfa-eddb-4778-8323-6733fa674d01 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:48.321060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5a7a-dace-4490-b4b4-34d0207a4c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:48.321699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5a7a-dace-4490-b4b4-34d0207a4c77 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.321974 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5a7a-dace-4490-b4b4-34d0207a4c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.322240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5a7a-dace-4490-b4b4-34d0207a4c77 None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.323404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e5a7a-dace-4490-b4b4-34d0207a4c77 None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:48.325032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e5a7a-dace-4490-b4b4-34d0207a4c77 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:48.330160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 190/375] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:48 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:48.330998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5a7a-dace-4490-b4b4-34d0207a4c77 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:48.359956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101ee64-4540-4846-bc35-e2602046ba54 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:48.388479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:48.399718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dda2e-7540-4d50-b191-98bf71c7b883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:48.400372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dda2e-7540-4d50-b191-98bf71c7b883 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.400680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dda2e-7540-4d50-b191-98bf71c7b883 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.400963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dda2e-7540-4d50-b191-98bf71c7b883 None admin] PATH_INFO: `/v3/users/a00b95b2003f4a48a96412ba05155923` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.402931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dda2e-7540-4d50-b191-98bf71c7b883 None admin] RBAC: Authorizing `identity:get_user(user_id=a00b95b2003f4a48a96412ba05155923)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:48.404683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dda2e-7540-4d50-b191-98bf71c7b883 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:48.411289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 186/376] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:00:48 2022] GET /identity/v3/users/a00b95b2003f4a48a96412ba05155923 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:48.412151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dda2e-7540-4d50-b191-98bf71c7b883 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:48.448469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5a7a-dace-4490-b4b4-34d0207a4c77 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:48.486060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994015ab-c752-46c1-87d8-1774e27884d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:48.486700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994015ab-c752-46c1-87d8-1774e27884d2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.486976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994015ab-c752-46c1-87d8-1774e27884d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.487243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994015ab-c752-46c1-87d8-1774e27884d2 None admin] PATH_INFO: `/v3/groups/89a5e1b32c9e4c078fe8c7f9919237ad` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.498152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994015ab-c752-46c1-87d8-1774e27884d2 None admin] RBAC: Authorizing `identity:get_group(group_id=89a5e1b32c9e4c078fe8c7f9919237ad)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:48.499633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994015ab-c752-46c1-87d8-1774e27884d2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:48.501518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 191/377] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:48 2022] GET /identity/v3/groups/89a5e1b32c9e4c078fe8c7f9919237ad => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:48.502510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc13e23-ebd6-402b-a5a9-22190da9a702 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.502790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc13e23-ebd6-402b-a5a9-22190da9a702 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.503077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc13e23-ebd6-402b-a5a9-22190da9a702 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.504392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 192/378] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:48.513367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc13e23-ebd6-402b-a5a9-22190da9a702 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:48.534169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dda2e-7540-4d50-b191-98bf71c7b883 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:48.556324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:48.582823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:48.584293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.585568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.586156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.587288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] RBAC: Adding query filter params (group.id=c4c39efb086f4a0f9f6a2386ea13ab16, role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=a74bec509592469588b2a87649030293) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:48.588173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:48.591017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:48.599758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 187/379] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:48 2022] GET /identity/v3/role_assignments?group.id=c4c39efb086f4a0f9f6a2386ea13ab16&scope.project.id=a74bec509592469588b2a87649030293&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:48.601042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:48.656652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc13e23-ebd6-402b-a5a9-22190da9a702 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:48.699389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be305a-64a5-47f4-996c-447958bbff7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:48.700404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be305a-64a5-47f4-996c-447958bbff7f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.700997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be305a-64a5-47f4-996c-447958bbff7f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.701471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be305a-64a5-47f4-996c-447958bbff7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.702464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be305a-64a5-47f4-996c-447958bbff7f None admin] RBAC: Adding query filter params (role.id=585290ba3e014e9e9ae58a7c1d1f1f67, user.id=a00b95b2003f4a48a96412ba05155923, scope.system=all) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:48.702965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be305a-64a5-47f4-996c-447958bbff7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:48.704806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be305a-64a5-47f4-996c-447958bbff7f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:48.709573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:48.723065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 193/380] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Aug 9 19:00:48 2022] GET /identity/v3/role_assignments?user.id=a00b95b2003f4a48a96412ba05155923&scope.system=all&role.id=585290ba3e014e9e9ae58a7c1d1f1f67 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:48.724735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2df12-052b-4806-9f2b-c442edc730bf None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.725240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2df12-052b-4806-9f2b-c442edc730bf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.725715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2df12-052b-4806-9f2b-c442edc730bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.746970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1fd2df12-052b-4806-9f2b-c442edc730bf None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:48.748605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1fd2df12-052b-4806-9f2b-c442edc730bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:48.787966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eacfa7-f462-4825-836c-6a0dbfdbe10b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:48.849126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 194/381] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:48.850747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124d3a5-c548-468c-a7c4-6b465f16e325 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.851284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124d3a5-c548-468c-a7c4-6b465f16e325 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.851869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124d3a5-c548-468c-a7c4-6b465f16e325 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.853789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 195/382] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:48.862029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf35fa-b3f1-4ec4-9ce2-55832ab25285 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.862682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf35fa-b3f1-4ec4-9ce2-55832ab25285 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.863159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf35fa-b3f1-4ec4-9ce2-55832ab25285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.893150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:48.893781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:48.894062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:48.894335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:48.895080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] RBAC: Adding query filter params (group.id=89a5e1b32c9e4c078fe8c7f9919237ad, role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=de422168330743c9990b96b3dab6b6f7) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:48.895390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:48.897498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-83bf35fa-b3f1-4ec4-9ce2-55832ab25285 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:48.898717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-83bf35fa-b3f1-4ec4-9ce2-55832ab25285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:48.902052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:48.914364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 188/383] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:48 2022] GET /identity/v3/role_assignments?group.id=89a5e1b32c9e4c078fe8c7f9919237ad&scope.project.id=de422168330743c9990b96b3dab6b6f7&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 260 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:48.915015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:48.939080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:49.083490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124901f-a33a-4fba-b279-75da116afc02 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:49.108853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:49.116260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 196/384] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:49.129747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22bc01-ba15-48b1-b57b-5dcb55307efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:49.130396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bc01-ba15-48b1-b57b-5dcb55307efd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:49.130675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bc01-ba15-48b1-b57b-5dcb55307efd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:49.130949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bc01-ba15-48b1-b57b-5dcb55307efd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:49.143606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22bc01-ba15-48b1-b57b-5dcb55307efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:49.148163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf35fa-b3f1-4ec4-9ce2-55832ab25285 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-fa22bc01-ba15-48b1-b57b-5dcb55307efd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:49.150451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:49.151373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:49.152188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:49.154316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 189/385] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:49.159703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22bc01-ba15-48b1-b57b-5dcb55307efd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:49.306513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22bc01-ba15-48b1-b57b-5dcb55307efd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:49.332652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf35fa-b3f1-4ec4-9ce2-55832ab25285 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:49.353337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:49.353891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:49.354181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:49.354479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:49.355596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:49.356062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:49.357596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:49.373119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 190/386] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:49.375936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:49.408384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dfc9b-152a-40d5-b1dd-72da0c97395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:49.409049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dfc9b-152a-40d5-b1dd-72da0c97395d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:49.409369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dfc9b-152a-40d5-b1dd-72da0c97395d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:49.409642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dfc9b-152a-40d5-b1dd-72da0c97395d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:49.432859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2dfc9b-152a-40d5-b1dd-72da0c97395d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:49.436608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7a2dfc9b-152a-40d5-b1dd-72da0c97395d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:49.439026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:49.439881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:49.440940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:49.444466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 197/387] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:49.448304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dfc9b-152a-40d5-b1dd-72da0c97395d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:49.472868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:49.511410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a0de3-3624-498f-be81-a8a0038a652a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:49.561023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe4cb9-2c84-4169-b2b4-00b8a7a9047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:49.561678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe4cb9-2c84-4169-b2b4-00b8a7a9047a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:49.561957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe4cb9-2c84-4169-b2b4-00b8a7a9047a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:49.562231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe4cb9-2c84-4169-b2b4-00b8a7a9047a None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:49.565843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe4cb9-2c84-4169-b2b4-00b8a7a9047a None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:49.567317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe4cb9-2c84-4169-b2b4-00b8a7a9047a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:49.568976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 191/388] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:49 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:49.575285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe4cb9-2c84-4169-b2b4-00b8a7a9047a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:49.628740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dfc9b-152a-40d5-b1dd-72da0c97395d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:49.681002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:49.681619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:49.681895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:49.682166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:49.683596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:49.684104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:49.685892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:49.699717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 198/389] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:49.706380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:49.806760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe4cb9-2c84-4169-b2b4-00b8a7a9047a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:49.829300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:49.844928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:49.884314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d7e0e-7a2f-41fc-8108-808298e0cd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:49.885344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d7e0e-7a2f-41fc-8108-808298e0cd69 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:49.886379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d7e0e-7a2f-41fc-8108-808298e0cd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:49.886852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d7e0e-7a2f-41fc-8108-808298e0cd69 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:49.889285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d7e0e-7a2f-41fc-8108-808298e0cd69 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:49.893138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d7e0e-7a2f-41fc-8108-808298e0cd69 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:49.895009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 192/390] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:49 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:49.901839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43825f8f-1b2d-48d6-9be9-00db49cbb031 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:49.902938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d7e0e-7a2f-41fc-8108-808298e0cd69 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:49.958678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0f94-1c81-4db8-8252-cca9bfe89670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:49.960222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0f94-1c81-4db8-8252-cca9bfe89670 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:49.960883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0f94-1c81-4db8-8252-cca9bfe89670 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:49.961356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0f94-1c81-4db8-8252-cca9bfe89670 None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:49.963038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a0f94-1c81-4db8-8252-cca9bfe89670 None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:49.965330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a0f94-1c81-4db8-8252-cca9bfe89670 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:49.967996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 199/391] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:49 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:49.974810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0f94-1c81-4db8-8252-cca9bfe89670 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:50.110319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d7e0e-7a2f-41fc-8108-808298e0cd69 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:50.129874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0f94-1c81-4db8-8252-cca9bfe89670 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:50.168145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece71dc-e905-458d-81bd-c0f0e732253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:50.169514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece71dc-e905-458d-81bd-c0f0e732253c None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:50.170489 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece71dc-e905-458d-81bd-c0f0e732253c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:50.171366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ec6a3-2793-44d6-8717-965eec983a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:50.172219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ec6a3-2793-44d6-8717-965eec983a09 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:50.172738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece71dc-e905-458d-81bd-c0f0e732253c None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e/groups/c4c39efb086f4a0f9f6a2386ea13ab16/roles/199fa373d2ff46b7a0c430b4a8c056e9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:50.173209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ec6a3-2793-44d6-8717-965eec983a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:50.173687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ec6a3-2793-44d6-8717-965eec983a09 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:50.176032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ec6a3-2793-44d6-8717-965eec983a09 None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:50.178799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece71dc-e905-458d-81bd-c0f0e732253c None admin] RBAC: Authorizing `identity:create_grant(project_id=57d897223df84e6ba0e4445299c6559e, group_id=c4c39efb086f4a0f9f6a2386ea13ab16, role_id=199fa373d2ff46b7a0c430b4a8c056e9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:50.180521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece71dc-e905-458d-81bd-c0f0e732253c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:50.180821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ec6a3-2793-44d6-8717-965eec983a09 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:50.182750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 200/392] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:49 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:50.189632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ec6a3-2793-44d6-8717-965eec983a09 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:50.205011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 193/393] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Aug 9 19:00:49 2022] PUT /identity/v3/projects/57d897223df84e6ba0e4445299c6559e/groups/c4c39efb086f4a0f9f6a2386ea13ab16/roles/199fa373d2ff46b7a0c430b4a8c056e9 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:00:50.216844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:50.319108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ec6a3-2793-44d6-8717-965eec983a09 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:50.357327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7ea82-1b02-4fbb-b1f9-aa7b1cad38c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:50.358011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ea82-1b02-4fbb-b1f9-aa7b1cad38c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:50.358287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ea82-1b02-4fbb-b1f9-aa7b1cad38c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:50.358561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ea82-1b02-4fbb-b1f9-aa7b1cad38c7 None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e/groups/c4c39efb086f4a0f9f6a2386ea13ab16/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:50.360536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7ea82-1b02-4fbb-b1f9-aa7b1cad38c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=57d897223df84e6ba0e4445299c6559e, group_id=c4c39efb086f4a0f9f6a2386ea13ab16, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:50.362049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7ea82-1b02-4fbb-b1f9-aa7b1cad38c7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:50.380863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 201/394] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Aug 9 19:00:50 2022] PUT /identity/v3/projects/57d897223df84e6ba0e4445299c6559e/groups/c4c39efb086f4a0f9f6a2386ea13ab16/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:50.664188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ae1ee-7d36-4e0c-a047-c1a02a4d086a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:50.664616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ae1ee-7d36-4e0c-a047-c1a02a4d086a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:50.664789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ae1ee-7d36-4e0c-a047-c1a02a4d086a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:50.666264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 194/395] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:50.672882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73d2d-49ff-496d-9f95-5b2cc0debc20 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:50.673517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73d2d-49ff-496d-9f95-5b2cc0debc20 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:50.674052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e73d2d-49ff-496d-9f95-5b2cc0debc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:50.693771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-81e73d2d-49ff-496d-9f95-5b2cc0debc20 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:50.695317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-81e73d2d-49ff-496d-9f95-5b2cc0debc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:50.763904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:50.774461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 202/396] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:50.811329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ae1ee-7d36-4e0c-a047-c1a02a4d086a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:50.882699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:50.961499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ae1ee-7d36-4e0c-a047-c1a02a4d086a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:51.011900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351babc-ba3b-42be-9f7d-edea35d26687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:51.012687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351babc-ba3b-42be-9f7d-edea35d26687 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.012965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351babc-ba3b-42be-9f7d-edea35d26687 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.013234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351babc-ba3b-42be-9f7d-edea35d26687 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.026859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351babc-ba3b-42be-9f7d-edea35d26687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e351babc-ba3b-42be-9f7d-edea35d26687 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:51.031537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:51.032400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:51.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 195/397] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:51.038743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e73d2d-49ff-496d-9f95-5b2cc0debc20 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:51.156354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ae470-3f5b-4824-ae2e-590cd006df36 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.156934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ae470-3f5b-4824-ae2e-590cd006df36 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.156934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ae470-3f5b-4824-ae2e-590cd006df36 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.158233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 196/398] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:51.163582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4119b-9d35-487e-b5e6-c8358399f10d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.163865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4119b-9d35-487e-b5e6-c8358399f10d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.164137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4119b-9d35-487e-b5e6-c8358399f10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.194047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c7f4119b-9d35-487e-b5e6-c8358399f10d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:51.195244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c7f4119b-9d35-487e-b5e6-c8358399f10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:51.213739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e73d2d-49ff-496d-9f95-5b2cc0debc20 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:51.256967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:51.257581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.257859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.258131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.258801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:51.259101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:51.260630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:51.270031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 203/399] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:51.275257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:51.319008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 197/400] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 19:00:51.344805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:51.360201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4119b-9d35-487e-b5e6-c8358399f10d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:51.404826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83499817-6c41-4e6c-a164-cf46a2a6ecee None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:51.438267 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:51.481185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b5068-dd9d-45c3-aa02-7773b0db7cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:51.481748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b5068-dd9d-45c3-aa02-7773b0db7cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.482024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b5068-dd9d-45c3-aa02-7773b0db7cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.482299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b5068-dd9d-45c3-aa02-7773b0db7cd9 None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.484310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b5068-dd9d-45c3-aa02-7773b0db7cd9 None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:51.485793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b5068-dd9d-45c3-aa02-7773b0db7cd9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:51.487388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 204/401] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:51 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:51.494523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b5068-dd9d-45c3-aa02-7773b0db7cd9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:51.556097 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4119b-9d35-487e-b5e6-c8358399f10d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:51.618506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df9bf7-54c2-4513-8703-70bc87d659cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:51.619694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df9bf7-54c2-4513-8703-70bc87d659cd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.622822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df9bf7-54c2-4513-8703-70bc87d659cd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.623325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df9bf7-54c2-4513-8703-70bc87d659cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.643085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df9bf7-54c2-4513-8703-70bc87d659cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-24df9bf7-54c2-4513-8703-70bc87d659cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:51.649291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:51.650162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:51.650948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:51.658969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b5068-dd9d-45c3-aa02-7773b0db7cd9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:51.662302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 198/402] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:51.667990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df9bf7-54c2-4513-8703-70bc87d659cd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:51.716859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:51.731753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59d045-e169-4d4c-a595-3a07059c0ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:51.732319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59d045-e169-4d4c-a595-3a07059c0ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.732632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59d045-e169-4d4c-a595-3a07059c0ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.732901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59d045-e169-4d4c-a595-3a07059c0ef0 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.735350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59d045-e169-4d4c-a595-3a07059c0ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:51.736829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59d045-e169-4d4c-a595-3a07059c0ef0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:51.746021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 205/403] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:51 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:51.747133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59d045-e169-4d4c-a595-3a07059c0ef0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:51.860074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:51.883441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df9bf7-54c2-4513-8703-70bc87d659cd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:51.927876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59d045-e169-4d4c-a595-3a07059c0ef0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:51.957237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:51.957863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.958144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.958421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.959101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:51.959517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:51.964470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:51.975106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 199/404] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:51.981958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:51.983254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16182858-3d17-4b48-b11b-6dbed005cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:51.983938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182858-3d17-4b48-b11b-6dbed005cfe6 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:51.984216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182858-3d17-4b48-b11b-6dbed005cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:51.984538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182858-3d17-4b48-b11b-6dbed005cfe6 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293/groups/c4c39efb086f4a0f9f6a2386ea13ab16/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:51.987060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16182858-3d17-4b48-b11b-6dbed005cfe6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a74bec509592469588b2a87649030293, group_id=c4c39efb086f4a0f9f6a2386ea13ab16, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:51.988533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16182858-3d17-4b48-b11b-6dbed005cfe6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:52.010487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 206/405] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Aug 9 19:00:51 2022] PUT /identity/v3/projects/a74bec509592469588b2a87649030293/groups/c4c39efb086f4a0f9f6a2386ea13ab16/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:52.043855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33002d73-2547-4bf9-9da4-866df7831f43 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.044208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33002d73-2547-4bf9-9da4-866df7831f43 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.044413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33002d73-2547-4bf9-9da4-866df7831f43 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.045839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 207/406] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:52.051490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ff2b-8fdd-4e3d-9170-bdcd2974cb57 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.051801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ff2b-8fdd-4e3d-9170-bdcd2974cb57 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.052043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ff2b-8fdd-4e3d-9170-bdcd2974cb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.093782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7b89ff2b-8fdd-4e3d-9170-bdcd2974cb57 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:52.095090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7b89ff2b-8fdd-4e3d-9170-bdcd2974cb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:52.127658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ee82f-f7d4-4a14-bd44-0f313b1f32d2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:52.174845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09892ca-9fc2-4f7e-8f11-2bae4347904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:52.176284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09892ca-9fc2-4f7e-8f11-2bae4347904b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.176785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09892ca-9fc2-4f7e-8f11-2bae4347904b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.177349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09892ca-9fc2-4f7e-8f11-2bae4347904b None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.178825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09892ca-9fc2-4f7e-8f11-2bae4347904b None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:52.180483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09892ca-9fc2-4f7e-8f11-2bae4347904b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:52.183420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 200/407] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:51 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:52.189479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09892ca-9fc2-4f7e-8f11-2bae4347904b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:52.227337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:52.235908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:52.249932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 208/408] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:52.277346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89ff2b-8fdd-4e3d-9170-bdcd2974cb57 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:52.315162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09892ca-9fc2-4f7e-8f11-2bae4347904b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:52.353831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de529-f87c-417e-ad2c-73a44d22242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:52.355150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de529-f87c-417e-ad2c-73a44d22242f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.355422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de529-f87c-417e-ad2c-73a44d22242f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.355691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26de529-f87c-417e-ad2c-73a44d22242f None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.356904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26de529-f87c-417e-ad2c-73a44d22242f None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:52.358352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26de529-f87c-417e-ad2c-73a44d22242f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:52.360346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 201/409] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:52 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:52.385148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de529-f87c-417e-ad2c-73a44d22242f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:52.398303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89ff2b-8fdd-4e3d-9170-bdcd2974cb57 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:52.445195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539d30-2f77-44f7-a4af-8cc83bda971a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:52.445831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539d30-2f77-44f7-a4af-8cc83bda971a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.446104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539d30-2f77-44f7-a4af-8cc83bda971a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.446375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539d30-2f77-44f7-a4af-8cc83bda971a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.471417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539d30-2f77-44f7-a4af-8cc83bda971a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-26539d30-2f77-44f7-a4af-8cc83bda971a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:52.476158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:52.477073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:52.477849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 209/410] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:52.488206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539d30-2f77-44f7-a4af-8cc83bda971a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:52.557412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:52.578099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:52.610499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26de529-f87c-417e-ad2c-73a44d22242f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:52.638446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539d30-2f77-44f7-a4af-8cc83bda971a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:52.675867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ba91-9c40-45cb-af1e-54219f1dbc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:52.677083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ba91-9c40-45cb-af1e-54219f1dbc53 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.678485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ba91-9c40-45cb-af1e-54219f1dbc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.679038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68ba91-9c40-45cb-af1e-54219f1dbc53 None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293/groups/c4c39efb086f4a0f9f6a2386ea13ab16/roles/199fa373d2ff46b7a0c430b4a8c056e9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.686123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68ba91-9c40-45cb-af1e-54219f1dbc53 None admin] RBAC: Authorizing `identity:create_grant(project_id=a74bec509592469588b2a87649030293, group_id=c4c39efb086f4a0f9f6a2386ea13ab16, role_id=199fa373d2ff46b7a0c430b4a8c056e9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:52.690433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68ba91-9c40-45cb-af1e-54219f1dbc53 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:52.715349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 202/411] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Aug 9 19:00:52 2022] PUT /identity/v3/projects/a74bec509592469588b2a87649030293/groups/c4c39efb086f4a0f9f6a2386ea13ab16/roles/199fa373d2ff46b7a0c430b4a8c056e9 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:52.716362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:52.717258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.717776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.718269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.719216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:52.719746 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:52.721433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:52.733374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 210/412] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:52.739100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ba91-9c40-45cb-af1e-54219f1dbc53 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:52.905212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68ba91-9c40-45cb-af1e-54219f1dbc53 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:52.953452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e891c-f0b8-426f-b65b-fab633f0aa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:52.954109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e891c-f0b8-426f-b65b-fab633f0aa58 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.954401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e891c-f0b8-426f-b65b-fab633f0aa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.954677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e891c-f0b8-426f-b65b-fab633f0aa58 None admin] PATH_INFO: `/v3/groups/89a5e1b32c9e4c078fe8c7f9919237ad` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.957210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e891c-f0b8-426f-b65b-fab633f0aa58 None admin] RBAC: Authorizing `identity:get_group(group_id=89a5e1b32c9e4c078fe8c7f9919237ad)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:52.958665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e891c-f0b8-426f-b65b-fab633f0aa58 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:52.960562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 203/413] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:52 2022] GET /identity/v3/groups/89a5e1b32c9e4c078fe8c7f9919237ad => generated 229 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:52.966328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:52.972052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818cbcf-1f9c-4e68-b097-6a8d2d66d2a6 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.972337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818cbcf-1f9c-4e68-b097-6a8d2d66d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.972640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818cbcf-1f9c-4e68-b097-6a8d2d66d2a6 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:52.973956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 204/414] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:52.984873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbba392-645f-482a-b41f-40a88877cc41 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:52.985181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbba392-645f-482a-b41f-40a88877cc41 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:52.985432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbba392-645f-482a-b41f-40a88877cc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:53.021597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-fdbba392-645f-482a-b41f-40a88877cc41 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:53.022741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fdbba392-645f-482a-b41f-40a88877cc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:53.043783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:53.103150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034f16e-da81-4f95-9de7-eb8ded2dc41a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:53.121661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:53.153110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e273736-c036-4698-9d30-c99c2a96e848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:53.153851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273736-c036-4698-9d30-c99c2a96e848 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:53.154128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273736-c036-4698-9d30-c99c2a96e848 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:53.154396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e273736-c036-4698-9d30-c99c2a96e848 None admin] PATH_INFO: `/v3/projects/de422168330743c9990b96b3dab6b6f7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:53.155528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e273736-c036-4698-9d30-c99c2a96e848 None admin] RBAC: Authorizing `identity:get_project(project_id=de422168330743c9990b96b3dab6b6f7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:53.157116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e273736-c036-4698-9d30-c99c2a96e848 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:53.159054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 211/415] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:52 2022] GET /identity/v3/projects/de422168330743c9990b96b3dab6b6f7 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:53.164680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e273736-c036-4698-9d30-c99c2a96e848 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:53.178732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 205/416] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:00:53.205462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c4e0-3c5a-493c-8479-d6d033a00c29 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:53.205762 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c4e0-3c5a-493c-8479-d6d033a00c29 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:53.206010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c4e0-3c5a-493c-8479-d6d033a00c29 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:53.207323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 206/417] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:00:53.218242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921785e1-af6f-4f45-9cbc-557932f92038 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:53.218523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921785e1-af6f-4f45-9cbc-557932f92038 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:53.218783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921785e1-af6f-4f45-9cbc-557932f92038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:53.237131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-921785e1-af6f-4f45-9cbc-557932f92038 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:53.238301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-921785e1-af6f-4f45-9cbc-557932f92038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:53.293022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e273736-c036-4698-9d30-c99c2a96e848 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:53.307591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 207/418] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:53.309264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921785e1-af6f-4f45-9cbc-557932f92038 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:53.331031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577061b4-f038-4951-b0c9-901722385e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:53.332026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577061b4-f038-4951-b0c9-901722385e9c None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:53.332536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577061b4-f038-4951-b0c9-901722385e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:53.333029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577061b4-f038-4951-b0c9-901722385e9c None admin] PATH_INFO: `/v3/projects/de422168330743c9990b96b3dab6b6f7/groups/89a5e1b32c9e4c078fe8c7f9919237ad/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:53.335111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577061b4-f038-4951-b0c9-901722385e9c None admin] RBAC: Authorizing `identity:create_grant(project_id=de422168330743c9990b96b3dab6b6f7, group_id=89a5e1b32c9e4c078fe8c7f9919237ad, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:53.336745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577061b4-f038-4951-b0c9-901722385e9c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:53.353594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 212/419] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Aug 9 19:00:53 2022] PUT /identity/v3/projects/de422168330743c9990b96b3dab6b6f7/groups/89a5e1b32c9e4c078fe8c7f9919237ad/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:00:53.361146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577061b4-f038-4951-b0c9-901722385e9c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:53.421634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:53.433776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:53.486856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921785e1-af6f-4f45-9cbc-557932f92038 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:53.554468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800223-5fd2-4208-be28-79b9a94c5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:53.555034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800223-5fd2-4208-be28-79b9a94c5f21 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:53.555309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800223-5fd2-4208-be28-79b9a94c5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:53.555577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800223-5fd2-4208-be28-79b9a94c5f21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:53.569939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577061b4-f038-4951-b0c9-901722385e9c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:53.580966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7800223-5fd2-4208-be28-79b9a94c5f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e7800223-5fd2-4208-be28-79b9a94c5f21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:53.586086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:53.586927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:53.587702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:53.590825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 208/420] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:53.597267 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800223-5fd2-4208-be28-79b9a94c5f21 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:53.638587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad2c1b-2074-406b-ad16-4b15ec7fe71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:53.639211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad2c1b-2074-406b-ad16-4b15ec7fe71b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:53.639485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad2c1b-2074-406b-ad16-4b15ec7fe71b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:53.639752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad2c1b-2074-406b-ad16-4b15ec7fe71b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:53.666367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad2c1b-2074-406b-ad16-4b15ec7fe71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b7ad2c1b-2074-406b-ad16-4b15ec7fe71b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:53.672337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:53.673247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:53.674024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:53.678727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 213/421] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:53.686924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad2c1b-2074-406b-ad16-4b15ec7fe71b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:53.724842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800223-5fd2-4208-be28-79b9a94c5f21 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:53.766121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:53.767890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:53.768928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:53.769983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:53.771075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:53.773375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:53.774944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:53.782693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:53.796804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 209/422] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:53.802385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:53.908579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:53.922034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad2c1b-2074-406b-ad16-4b15ec7fe71b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:53.975707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dded46f-6be2-4dbe-97d6-d13ba2b74441 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:54.015452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:54.016728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:54.017006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:54.017283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:54.024461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:54.024461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:54.024461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:54.030123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 214/423] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:54.033165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b9962-122d-4b2a-b3ed-f75c8d0c6a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:54.033799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b9962-122d-4b2a-b3ed-f75c8d0c6a70 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:54.034084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b9962-122d-4b2a-b3ed-f75c8d0c6a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:54.034362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b9962-122d-4b2a-b3ed-f75c8d0c6a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:54.041178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:54.176996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:54.209361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b9962-122d-4b2a-b3ed-f75c8d0c6a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:54.210847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b9962-122d-4b2a-b3ed-f75c8d0c6a70 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:54.225060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a581b5-fdb2-453e-92c7-f0751eade0ad None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:54.274926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142df7a-8217-4c79-a79c-2b55674f098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:54.275970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142df7a-8217-4c79-a79c-2b55674f098c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:54.276357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142df7a-8217-4c79-a79c-2b55674f098c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:54.276861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142df7a-8217-4c79-a79c-2b55674f098c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:54.312147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:54.357807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142df7a-8217-4c79-a79c-2b55674f098c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:54.359406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142df7a-8217-4c79-a79c-2b55674f098c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:54.461855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 210/424] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 19:00:54.467994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b9962-122d-4b2a-b3ed-f75c8d0c6a70 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:54.485513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 215/425] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:54.491001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142df7a-8217-4c79-a79c-2b55674f098c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:54.608897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:54.617102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b9962-122d-4b2a-b3ed-f75c8d0c6a70 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:54.619872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:54.680626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd783c-8bb9-4616-8cdf-351c69ead51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:54.681541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd783c-8bb9-4616-8cdf-351c69ead51f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:54.681822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd783c-8bb9-4616-8cdf-351c69ead51f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:54.682520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd783c-8bb9-4616-8cdf-351c69ead51f None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:54.683912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd783c-8bb9-4616-8cdf-351c69ead51f None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:54.685392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142df7a-8217-4c79-a79c-2b55674f098c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:54.685635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd783c-8bb9-4616-8cdf-351c69ead51f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:54.688804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 211/426] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:54 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:54.690012 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb52c5-148f-4d2c-89f6-ff2176bec283 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:54.690292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb52c5-148f-4d2c-89f6-ff2176bec283 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:54.690566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb52c5-148f-4d2c-89f6-ff2176bec283 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:54.692023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 212/427] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:54.692744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb52c5-148f-4d2c-89f6-ff2176bec283 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:54.733673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94924f99-76fc-4394-9a2e-14f8abe386db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:54.734171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924f99-76fc-4394-9a2e-14f8abe386db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:54.734454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924f99-76fc-4394-9a2e-14f8abe386db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:54.734731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94924f99-76fc-4394-9a2e-14f8abe386db None admin] PATH_INFO: `/v3/projects/57d897223df84e6ba0e4445299c6559e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:54.735863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94924f99-76fc-4394-9a2e-14f8abe386db None admin] RBAC: Authorizing `identity:get_project(project_id=57d897223df84e6ba0e4445299c6559e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:54.737355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94924f99-76fc-4394-9a2e-14f8abe386db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:54.743506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 216/428] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:54 2022] GET /identity/v3/projects/57d897223df84e6ba0e4445299c6559e => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:54.744791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4cb00-6f3f-4c82-aab6-eb379da67b89 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:54.745183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4cb00-6f3f-4c82-aab6-eb379da67b89 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:54.745460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4cb00-6f3f-4c82-aab6-eb379da67b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:54.766804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cfb4cb00-6f3f-4c82-aab6-eb379da67b89 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:54.768191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cfb4cb00-6f3f-4c82-aab6-eb379da67b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:54.818008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb52c5-148f-4d2c-89f6-ff2176bec283 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:54.856421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e2bc6-335c-483b-af2b-10377eba663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:54.857161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e2bc6-335c-483b-af2b-10377eba663a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:54.857442 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e2bc6-335c-483b-af2b-10377eba663a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:54.857703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e2bc6-335c-483b-af2b-10377eba663a None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:54.858869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e2bc6-335c-483b-af2b-10377eba663a None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:54.860290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e2bc6-335c-483b-af2b-10377eba663a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:54.860686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 217/429] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:54.861581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4cb00-6f3f-4c82-aab6-eb379da67b89 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:54.862032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 213/430] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:54 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:54.872522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e2bc6-335c-483b-af2b-10377eba663a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:54.928309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:54.935016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:54.985966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4cb00-6f3f-4c82-aab6-eb379da67b89 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:55.006139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e2bc6-335c-483b-af2b-10377eba663a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:55.024779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb5f25-f91e-4b52-b3b8-065d2c0beedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:55.025781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb5f25-f91e-4b52-b3b8-065d2c0beedf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.026296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb5f25-f91e-4b52-b3b8-065d2c0beedf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.027293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb5f25-f91e-4b52-b3b8-065d2c0beedf None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.028927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb5f25-f91e-4b52-b3b8-065d2c0beedf None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:55.031389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb5f25-f91e-4b52-b3b8-065d2c0beedf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:55.034320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 218/431] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:54 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:55.035585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb5f25-f91e-4b52-b3b8-065d2c0beedf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:55.042346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:55.043122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.043414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.043667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.044396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] RBAC: Adding query filter params (group.id=c4c39efb086f4a0f9f6a2386ea13ab16, role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=57d897223df84e6ba0e4445299c6559e) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:55.044744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:55.046163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:55.060620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 214/432] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:54 2022] GET /identity/v3/role_assignments?group.id=c4c39efb086f4a0f9f6a2386ea13ab16&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:55.061647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:55.170105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb5f25-f91e-4b52-b3b8-065d2c0beedf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:55.208611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:55.224953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:55.228256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a443da-c970-456e-8a23-4db9f0eb80f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:55.230795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a443da-c970-456e-8a23-4db9f0eb80f5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.231716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a443da-c970-456e-8a23-4db9f0eb80f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.232183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a443da-c970-456e-8a23-4db9f0eb80f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.233436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8406038-d220-450c-8a39-8a644b8fa6d2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:55.249117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a443da-c970-456e-8a23-4db9f0eb80f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-62a443da-c970-456e-8a23-4db9f0eb80f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:55.255174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:55.256078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 19:00:55.256908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:00:55.259143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 219/433] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:55.260705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467520-fbd0-49e2-bad0-911e891d0aac None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.261026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467520-fbd0-49e2-bad0-911e891d0aac None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.261292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467520-fbd0-49e2-bad0-911e891d0aac None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.262804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 220/434] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:55.265477 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24467520-fbd0-49e2-bad0-911e891d0aac None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:55.282678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cf6d9-aec8-4f57-b599-06eecc9c85c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:55.283410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf6d9-aec8-4f57-b599-06eecc9c85c6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.283687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf6d9-aec8-4f57-b599-06eecc9c85c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.283968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf6d9-aec8-4f57-b599-06eecc9c85c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.284758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cf6d9-aec8-4f57-b599-06eecc9c85c6 None admin] RBAC: Adding query filter params (group.id=c4c39efb086f4a0f9f6a2386ea13ab16, role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=57d897223df84e6ba0e4445299c6559e) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:55.285156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cf6d9-aec8-4f57-b599-06eecc9c85c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:55.286591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cf6d9-aec8-4f57-b599-06eecc9c85c6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:55.294368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 215/435] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:55 2022] GET /identity/v3/role_assignments?group.id=c4c39efb086f4a0f9f6a2386ea13ab16&scope.project.id=57d897223df84e6ba0e4445299c6559e&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 611 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:55.295719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ed567-8f6a-4d2a-b888-45d3e1fd6b61 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.295874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ed567-8f6a-4d2a-b888-45d3e1fd6b61 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.296126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ed567-8f6a-4d2a-b888-45d3e1fd6b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.316727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1f7ed567-8f6a-4d2a-b888-45d3e1fd6b61 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:55.318320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1f7ed567-8f6a-4d2a-b888-45d3e1fd6b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:55.409576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 216/436] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:55.416629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24467520-fbd0-49e2-bad0-911e891d0aac None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:55.436425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ed567-8f6a-4d2a-b888-45d3e1fd6b61 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:55.462670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:55.463294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.463574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.464394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.465508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:55.465816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:55.468124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:55.481560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 221/437] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:00:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:55.486832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:55.554982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:55.563748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:55.578815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ed567-8f6a-4d2a-b888-45d3e1fd6b61 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:55.629910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869db00-a462-4db7-a0b9-7273bb48cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:55.629910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869db00-a462-4db7-a0b9-7273bb48cb09 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.629910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869db00-a462-4db7-a0b9-7273bb48cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.629910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869db00-a462-4db7-a0b9-7273bb48cb09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.633117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d88c4-95b7-44dc-bf89-b847e3cc34f3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:55.642232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869db00-a462-4db7-a0b9-7273bb48cb09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a869db00-a462-4db7-a0b9-7273bb48cb09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:55.647354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:55.648474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:55.649604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 217/438] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:00:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:00:55.654977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869db00-a462-4db7-a0b9-7273bb48cb09 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:55.680564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8fe89-5aa6-4792-936b-b848623f6ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:55.681425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8fe89-5aa6-4792-936b-b848623f6ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.681777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8fe89-5aa6-4792-936b-b848623f6ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.682122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8fe89-5aa6-4792-936b-b848623f6ac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.766201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8fe89-5aa6-4792-936b-b848623f6ac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:55.769337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8fe89-5aa6-4792-936b-b848623f6ac4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:55.806024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869db00-a462-4db7-a0b9-7273bb48cb09 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:55.828334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:55.845701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6ff16-00ce-47e8-92ae-f61fd82a4b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:55.846310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ff16-00ce-47e8-92ae-f61fd82a4b3d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.846588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ff16-00ce-47e8-92ae-f61fd82a4b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.846861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ff16-00ce-47e8-92ae-f61fd82a4b3d None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.847530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6ff16-00ce-47e8-92ae-f61fd82a4b3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:55.847838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6ff16-00ce-47e8-92ae-f61fd82a4b3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:55.849257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6ff16-00ce-47e8-92ae-f61fd82a4b3d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:55.858457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 218/439] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:00:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:55.859909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19f4a7-5ce5-4e69-9213-391f7042986a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.860292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19f4a7-5ce5-4e69-9213-391f7042986a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.860705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19f4a7-5ce5-4e69-9213-391f7042986a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.862357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 219/440] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:55.867612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046c89-1cfe-449c-a110-d83cd5f82ada None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:55.868014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046c89-1cfe-449c-a110-d83cd5f82ada None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:55.868399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046c89-1cfe-449c-a110-d83cd5f82ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:55.893035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-87046c89-1cfe-449c-a110-d83cd5f82ada None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:55.894217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-87046c89-1cfe-449c-a110-d83cd5f82ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:55.903840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:55.905541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 222/441] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 19:00:55.906770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8fe89-5aa6-4792-936b-b848623f6ac4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:55.973490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 220/442] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:55.974339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87046c89-1cfe-449c-a110-d83cd5f82ada None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.019548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8fe89-5aa6-4792-936b-b848623f6ac4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.053532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415891e-47d9-4beb-b4c4-bd7f89efd905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.054291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415891e-47d9-4beb-b4c4-bd7f89efd905 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.054565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415891e-47d9-4beb-b4c4-bd7f89efd905 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.054873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415891e-47d9-4beb-b4c4-bd7f89efd905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.070710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:56.075969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87046c89-1cfe-449c-a110-d83cd5f82ada None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.110668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1ac1-e1e7-4434-824b-94a83520d7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.111295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1ac1-e1e7-4434-824b-94a83520d7ff None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.111572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1ac1-e1e7-4434-824b-94a83520d7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.111843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1ac1-e1e7-4434-824b-94a83520d7ff None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.112978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e1ac1-e1e7-4434-824b-94a83520d7ff None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.114403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e1ac1-e1e7-4434-824b-94a83520d7ff None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.116009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 221/443] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:55 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:56.116777 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1ac1-e1e7-4434-824b-94a83520d7ff None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.123253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415891e-47d9-4beb-b4c4-bd7f89efd905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.124745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415891e-47d9-4beb-b4c4-bd7f89efd905 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.129422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:56.219493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1ac1-e1e7-4434-824b-94a83520d7ff None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.222416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 223/444] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:56.223241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415891e-47d9-4beb-b4c4-bd7f89efd905 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.253943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32472da-9b98-4050-a00c-a75eed19a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.254562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32472da-9b98-4050-a00c-a75eed19a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.254843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32472da-9b98-4050-a00c-a75eed19a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.255116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32472da-9b98-4050-a00c-a75eed19a6c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.265691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32472da-9b98-4050-a00c-a75eed19a6c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e32472da-9b98-4050-a00c-a75eed19a6c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:00:56.270390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:00:56.271226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:00:56.272002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:00:56.272612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:56.272681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 222/445] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:00:56.272681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32472da-9b98-4050-a00c-a75eed19a6c0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.330185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415891e-47d9-4beb-b4c4-bd7f89efd905 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.341719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:56.363737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a821d86-13a9-4009-addd-681255e7c3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.364350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821d86-13a9-4009-addd-681255e7c3ba None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.364655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821d86-13a9-4009-addd-681255e7c3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.364922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821d86-13a9-4009-addd-681255e7c3ba None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.365999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a821d86-13a9-4009-addd-681255e7c3ba None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.367424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a821d86-13a9-4009-addd-681255e7c3ba None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.369287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 224/446] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:56.370655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a821d86-13a9-4009-addd-681255e7c3ba None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.373161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32472da-9b98-4050-a00c-a75eed19a6c0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.410169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c47372-5dd5-4978-8b07-ff9d2d395e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.410796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47372-5dd5-4978-8b07-ff9d2d395e3a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.411065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47372-5dd5-4978-8b07-ff9d2d395e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.411332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47372-5dd5-4978-8b07-ff9d2d395e3a None admin] PATH_INFO: `/v3/projects/a74bec509592469588b2a87649030293` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.412460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c47372-5dd5-4978-8b07-ff9d2d395e3a None admin] RBAC: Authorizing `identity:get_project(project_id=a74bec509592469588b2a87649030293)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.413909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c47372-5dd5-4978-8b07-ff9d2d395e3a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.415507 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 223/447] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/projects/a74bec509592469588b2a87649030293 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:56.416245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c47372-5dd5-4978-8b07-ff9d2d395e3a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.475149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a821d86-13a9-4009-addd-681255e7c3ba None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.490177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:56.510851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.511441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.511712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.512234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.512662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:56.512957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.514313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.518030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c47372-5dd5-4978-8b07-ff9d2d395e3a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.522302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 225/448] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:56.523074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.552761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.553369 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.553647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.553920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.554647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] RBAC: Adding query filter params (group.id=c4c39efb086f4a0f9f6a2386ea13ab16, role.id=46921b1a8b1a4d2d9ad0dd7070581618, scope.project.id=a74bec509592469588b2a87649030293) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:56.554960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.556378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.563256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 224/449] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/role_assignments?group.id=c4c39efb086f4a0f9f6a2386ea13ab16&scope.project.id=a74bec509592469588b2a87649030293&role.id=46921b1a8b1a4d2d9ad0dd7070581618 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:56.564034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.566634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:56.647481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494d8aa-14e2-4369-af9b-1346bd5b4794 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.686993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40947d10-a541-43d0-ac86-363e7a4ecf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.687629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40947d10-a541-43d0-ac86-363e7a4ecf9c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.687906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40947d10-a541-43d0-ac86-363e7a4ecf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.688121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93e652-6188-4564-99bf-565ab0b115c8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.688222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40947d10-a541-43d0-ac86-363e7a4ecf9c None admin] PATH_INFO: `/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.689336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40947d10-a541-43d0-ac86-363e7a4ecf9c None admin] RBAC: Authorizing `identity:get_group(group_id=c4c39efb086f4a0f9f6a2386ea13ab16)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.690790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40947d10-a541-43d0-ac86-363e7a4ecf9c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.692424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 226/450] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/groups/c4c39efb086f4a0f9f6a2386ea13ab16 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:56.698947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40947d10-a541-43d0-ac86-363e7a4ecf9c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:56.731815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349970c0-6835-441e-ad65-a9ce672c48d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.732497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349970c0-6835-441e-ad65-a9ce672c48d2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.733934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349970c0-6835-441e-ad65-a9ce672c48d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.734211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349970c0-6835-441e-ad65-a9ce672c48d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.756631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:56.822734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40947d10-a541-43d0-ac86-363e7a4ecf9c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:56.825960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349970c0-6835-441e-ad65-a9ce672c48d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.827448 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349970c0-6835-441e-ad65-a9ce672c48d2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.829487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:56.862338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:56.863851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:56.863851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:56.863851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:56.864200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] RBAC: Adding query filter params (group.id=c4c39efb086f4a0f9f6a2386ea13ab16, role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=a74bec509592469588b2a87649030293) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:56.864535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:56.865971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:56.873465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 227/451] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/role_assignments?group.id=c4c39efb086f4a0f9f6a2386ea13ab16&scope.project.id=a74bec509592469588b2a87649030293&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:00:56.951014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 225/452] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:00:56.957070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:57.065308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb472d-dff0-4456-85fb-34f93b4b6d38 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:57.074852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:57.102208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcb71d-239c-49f9-8eba-d6f0de9ca68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:57.102837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcb71d-239c-49f9-8eba-d6f0de9ca68b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:57.103111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcb71d-239c-49f9-8eba-d6f0de9ca68b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:57.103405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcb71d-239c-49f9-8eba-d6f0de9ca68b None admin] PATH_INFO: `/v3/projects/de422168330743c9990b96b3dab6b6f7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:57.104570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcb71d-239c-49f9-8eba-d6f0de9ca68b None admin] RBAC: Authorizing `identity:get_project(project_id=de422168330743c9990b96b3dab6b6f7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:57.105979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcb71d-239c-49f9-8eba-d6f0de9ca68b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:57.107582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 228/453] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:00:56 2022] GET /identity/v3/projects/de422168330743c9990b96b3dab6b6f7 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:57.113243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349970c0-6835-441e-ad65-a9ce672c48d2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:57.217555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349970c0-6835-441e-ad65-a9ce672c48d2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:57.219033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:57.252908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715288f-bb12-4caf-906e-95a768877c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:57.253529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715288f-bb12-4caf-906e-95a768877c35 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:57.253802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715288f-bb12-4caf-906e-95a768877c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:57.254072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715288f-bb12-4caf-906e-95a768877c35 None admin] PATH_INFO: `/v3/groups/89a5e1b32c9e4c078fe8c7f9919237ad` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:57.255146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715288f-bb12-4caf-906e-95a768877c35 None admin] RBAC: Authorizing `identity:get_group(group_id=89a5e1b32c9e4c078fe8c7f9919237ad)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:57.256588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715288f-bb12-4caf-906e-95a768877c35 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:57.258139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 226/454] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Aug 9 19:00:57 2022] GET /identity/v3/groups/89a5e1b32c9e4c078fe8c7f9919237ad => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:57.264013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcb71d-239c-49f9-8eba-d6f0de9ca68b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:00:57.360269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcb71d-239c-49f9-8eba-d6f0de9ca68b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:00:57.394606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e78d2-9482-4a86-b4a5-5b0c2960e969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:00:57.395208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e78d2-9482-4a86-b4a5-5b0c2960e969 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:57.395491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e78d2-9482-4a86-b4a5-5b0c2960e969 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:57.395766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e78d2-9482-4a86-b4a5-5b0c2960e969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:57.396522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e78d2-9482-4a86-b4a5-5b0c2960e969 None admin] RBAC: Adding query filter params (group.id=89a5e1b32c9e4c078fe8c7f9919237ad, role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=de422168330743c9990b96b3dab6b6f7) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:00:57.396834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e78d2-9482-4a86-b4a5-5b0c2960e969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:00:57.398267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e78d2-9482-4a86-b4a5-5b0c2960e969 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:00:57.404678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 229/455] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Aug 9 19:00:57 2022] GET /identity/v3/role_assignments?group.id=89a5e1b32c9e4c078fe8c7f9919237ad&scope.project.id=de422168330743c9990b96b3dab6b6f7&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:00:59.810558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f4dca-5a46-47ac-a3b7-990abfe4dd62 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:59.811071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f4dca-5a46-47ac-a3b7-990abfe4dd62 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:59.811071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f4dca-5a46-47ac-a3b7-990abfe4dd62 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:59.812677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 227/456] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:59.818186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f05ef9-d548-4e99-b8fb-1639d2863674 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:59.818604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f05ef9-d548-4e99-b8fb-1639d2863674 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:59.818776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f05ef9-d548-4e99-b8fb-1639d2863674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:59.841750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-05f05ef9-d548-4e99-b8fb-1639d2863674 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:59.842950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-05f05ef9-d548-4e99-b8fb-1639d2863674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:59.890937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:00:59.922523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307202-3f01-4d78-b685-d5871c36f7a8 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:59.922843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307202-3f01-4d78-b685-d5871c36f7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:59.923086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50307202-3f01-4d78-b685-d5871c36f7a8 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:59.924681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 228/457] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:00:59.929964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e2c74-884d-44a3-a2d3-4618b618584d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:00:59.930489 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e2c74-884d-44a3-a2d3-4618b618584d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:00:59.930970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e2c74-884d-44a3-a2d3-4618b618584d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:00:59.954286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 230/458] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:00:59.959265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-296e2c74-884d-44a3-a2d3-4618b618584d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:00:59.960488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-296e2c74-884d-44a3-a2d3-4618b618584d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:00:59.978200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f05ef9-d548-4e99-b8fb-1639d2863674 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:00.043743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 229/459] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:00.079709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e2c74-884d-44a3-a2d3-4618b618584d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:00.128372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f05ef9-d548-4e99-b8fb-1639d2863674 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:00.177013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20c8da-6ac6-41e8-91b6-351e2379c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:00.177894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c8da-6ac6-41e8-91b6-351e2379c092 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.178318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c8da-6ac6-41e8-91b6-351e2379c092 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.178740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c8da-6ac6-41e8-91b6-351e2379c092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.196876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20c8da-6ac6-41e8-91b6-351e2379c092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:00.199842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20c8da-6ac6-41e8-91b6-351e2379c092 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0b20c8da-6ac6-41e8-91b6-351e2379c092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:00.224723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:00.225688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:00.226514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:00.226514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:00.226514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:00.232724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 231/460] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:00.234899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f93a9f-f271-4c10-b5a5-374372666627 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.235321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f93a9f-f271-4c10-b5a5-374372666627 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.235804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f93a9f-f271-4c10-b5a5-374372666627 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.237072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:00.239925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 232/461] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:00.240828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e2c74-884d-44a3-a2d3-4618b618584d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:00.246529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f93a9f-f271-4c10-b5a5-374372666627 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:00.279900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:00.306027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973619a-0c69-431e-a38c-049829d88156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:00.306954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973619a-0c69-431e-a38c-049829d88156 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.307233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973619a-0c69-431e-a38c-049829d88156 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.308251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973619a-0c69-431e-a38c-049829d88156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.323294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973619a-0c69-431e-a38c-049829d88156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:00.325891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973619a-0c69-431e-a38c-049829d88156 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2973619a-0c69-431e-a38c-049829d88156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:00.345302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:00.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:00.347041 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:00.347041 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:00.347041 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:00.347590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 230/462] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:00.348527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d919-04d6-4055-ad1e-278ba6140452 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.348813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d919-04d6-4055-ad1e-278ba6140452 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.349089 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d919-04d6-4055-ad1e-278ba6140452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.376837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d059d919-04d6-4055-ad1e-278ba6140452 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:00.378089 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d059d919-04d6-4055-ad1e-278ba6140452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:00.378664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f93a9f-f271-4c10-b5a5-374372666627 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:00.429304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:00.430188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.430619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.431033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.431819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:00.432414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:00.434049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:00.454050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 233/463] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:00.455206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:00.464127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 231/464] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:00.468801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059d919-04d6-4055-ad1e-278ba6140452 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:00.562373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:00.584787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059d919-04d6-4055-ad1e-278ba6140452 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:00.590382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:00.617999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1e0c6-8d78-4324-a9b0-48cc5fbb4028 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:00.626281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9055b-b160-49d6-b24c-8d49b4f2ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:00.627127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9055b-b160-49d6-b24c-8d49b4f2ddf4 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.627568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9055b-b160-49d6-b24c-8d49b4f2ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.628025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9055b-b160-49d6-b24c-8d49b4f2ddf4 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.628935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9055b-b160-49d6-b24c-8d49b4f2ddf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:00.630592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9055b-b160-49d6-b24c-8d49b4f2ddf4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:00.657872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 232/465] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:01:00 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:00.659045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9055b-b160-49d6-b24c-8d49b4f2ddf4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:00.659184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:00.659881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.660159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.660432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.661107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:00.661461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:00.663182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:00.680104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 234/466] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:00.686231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:00.820050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783b59b-790e-4197-a970-54eff3ac24b2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:00.864958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210fba3-8a14-4df1-9cb2-4464b525c097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:00.865579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210fba3-8a14-4df1-9cb2-4464b525c097 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.865858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210fba3-8a14-4df1-9cb2-4464b525c097 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.866135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210fba3-8a14-4df1-9cb2-4464b525c097 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.866782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210fba3-8a14-4df1-9cb2-4464b525c097 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:00.868307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210fba3-8a14-4df1-9cb2-4464b525c097 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:00.871288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9055b-b160-49d6-b24c-8d49b4f2ddf4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:00.895656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:00.904214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 235/467] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:01:00 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:00.906504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a5f83-84bd-4962-94da-f15a51db2926 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.907967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a5f83-84bd-4962-94da-f15a51db2926 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.908247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a5f83-84bd-4962-94da-f15a51db2926 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 236/468] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:00.919310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6496f7-9717-4228-a7e5-2caa6fadb738 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.919615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6496f7-9717-4228-a7e5-2caa6fadb738 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.919853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6496f7-9717-4228-a7e5-2caa6fadb738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.924145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:00.924840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:00.925167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:00.925490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:00.937154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:00.942373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:00.944079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:00.952383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6a6496f7-9717-4228-a7e5-2caa6fadb738 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:00.953964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6a6496f7-9717-4228-a7e5-2caa6fadb738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:00.963389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:00.965486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:00.966320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:00.966320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:00.966320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:00.970353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 233/469] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:00.976813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:01.064467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 237/470] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:01:01.092413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6496f7-9717-4228-a7e5-2caa6fadb738 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:01.139552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afdf07-8e3b-4af1-b9fb-235bdb301d93 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:01.177409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:01.178259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:01.178694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:01.179109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:01.179897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:01.180342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:01.181863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:01.196486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 234/471] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:01.205548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:01.237618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6496f7-9717-4228-a7e5-2caa6fadb738 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:01.265325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:01.265832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:01.275135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:01.275938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:01.277048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:01.277333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:01.288982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:01.290494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:01.307019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:01.308801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:01.309615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:01.309615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:01.309615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:01.312577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 238/472] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:01.315116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:01.328783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7c289-8412-4f5a-b88c-3685c6ee0ae9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:01.374541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd940e39-c9fc-4ed5-940f-3c2f3cd697f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:01.375521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd940e39-c9fc-4ed5-940f-3c2f3cd697f3 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:01.376000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd940e39-c9fc-4ed5-940f-3c2f3cd697f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:01.376517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd940e39-c9fc-4ed5-940f-3c2f3cd697f3 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:01.377370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd940e39-c9fc-4ed5-940f-3c2f3cd697f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:01.378983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd940e39-c9fc-4ed5-940f-3c2f3cd697f3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:01.427304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 235/473] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:01:01 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:01.439153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01a17d-6d0a-46f5-be5e-dcb8636eb072 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:01.462782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be65da0-4b18-45e8-8827-c24e236edbcb None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:01.463408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be65da0-4b18-45e8-8827-c24e236edbcb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:01.463956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be65da0-4b18-45e8-8827-c24e236edbcb None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:01.465998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 236/474] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:01:01.471889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf1d47-32ba-4ce2-a485-bbb9640aa495 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:01.472571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf1d47-32ba-4ce2-a485-bbb9640aa495 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:01.473117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf1d47-32ba-4ce2-a485-bbb9640aa495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:01.479725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:01.480385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:01.480701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:01.480975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:01.481684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:01.481997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:01.483399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:01.494209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-84bf1d47-32ba-4ce2-a485-bbb9640aa495 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:01.496988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-84bf1d47-32ba-4ce2-a485-bbb9640aa495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:01.501804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 239/475] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:01.503951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:01.553139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:01.619929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64818450-9fa0-4bf1-9f61-e280dd39f347 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:01.644615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 237/476] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:01:01.668308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed545a2e-2d7b-472e-949b-ef24e7666968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:01.669269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed545a2e-2d7b-472e-949b-ef24e7666968 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:01.669693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed545a2e-2d7b-472e-949b-ef24e7666968 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:01.670108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed545a2e-2d7b-472e-949b-ef24e7666968 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:01.670934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed545a2e-2d7b-472e-949b-ef24e7666968 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:01.672704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed545a2e-2d7b-472e-949b-ef24e7666968 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:01.687280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf1d47-32ba-4ce2-a485-bbb9640aa495 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:01.704615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 240/477] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:01:01 2022] POST /identity/v3/users => generated 249 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:01.745732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:01.838150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf1d47-32ba-4ce2-a485-bbb9640aa495 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:01.878927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:01.879700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:01.879871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:01.880194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:01.881037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:01.882690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:01.894113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.sql.core [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 19:01:01.894113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 19:01:01.894113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [parameters: {'id': 'eb34acf1f3ff4fe7aa0f3848b4a95608', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 19:01:01.894113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80338) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application conn.query(q) Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result.read() Aug 09 19:01:01.911742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return ref.to_dict() Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 19:01:01.912679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application next(self.gen) Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application yield resource Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application next(self.gen) Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise self.value Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application session.commit() Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 19:01:01.913558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.session.flush() Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application compat.raise_( Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 19:01:01.914389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 19:01:01.915249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application conn.query(q) Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result.read() Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 19:01:01.916038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [parameters: {'id': 'eb34acf1f3ff4fe7aa0f3848b4a95608', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:01.917749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 19:01:01.918682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:01.919577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 238/478] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Aug 9 19:01:01 2022] POST /identity/v3/roles => generated 144 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:01.926135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed545a2e-2d7b-472e-949b-ef24e7666968 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:02.035121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:02.041341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed545a2e-2d7b-472e-949b-ef24e7666968 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:02.079549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbff19-d145-44d6-9f53-6ca07f18afc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:02.080185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbff19-d145-44d6-9f53-6ca07f18afc8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:02.080480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbff19-d145-44d6-9f53-6ca07f18afc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:02.080759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbff19-d145-44d6-9f53-6ca07f18afc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:02.092495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbff19-d145-44d6-9f53-6ca07f18afc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1bcbff19-d145-44d6-9f53-6ca07f18afc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:02.098585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:02.099436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:02.100401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 241/479] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:02.107742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:02.214998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:02.217998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced830-045a-41cd-bd67-f8297779b2be None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:02.257159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5fcd3-b255-4ea8-be70-ab6fef5631b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:02.257830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5fcd3-b255-4ea8-be70-ab6fef5631b0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:02.258121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5fcd3-b255-4ea8-be70-ab6fef5631b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:02.258394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5fcd3-b255-4ea8-be70-ab6fef5631b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:02.259070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5fcd3-b255-4ea8-be70-ab6fef5631b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:02.259374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5fcd3-b255-4ea8-be70-ab6fef5631b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:02.261322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5fcd3-b255-4ea8-be70-ab6fef5631b0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:02.270351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 239/480] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:03.427545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd28b9-d852-49de-b700-8df8fa156099 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:03.428045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd28b9-d852-49de-b700-8df8fa156099 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:03.428120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd28b9-d852-49de-b700-8df8fa156099 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:03.429446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 242/481] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:03.435383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab6062-6cbd-4434-9844-9b9c0f180731 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:03.435814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab6062-6cbd-4434-9844-9b9c0f180731 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:03.435911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab6062-6cbd-4434-9844-9b9c0f180731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:03.460065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-70ab6062-6cbd-4434-9844-9b9c0f180731 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:03.461819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-70ab6062-6cbd-4434-9844-9b9c0f180731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:03.473832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447515a1-dd76-4333-815e-321edecc741e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:03.474107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447515a1-dd76-4333-815e-321edecc741e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:03.474370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447515a1-dd76-4333-815e-321edecc741e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:03.475915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 243/482] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:03.490126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cd3d3-0b28-4642-85b3-a590ee91ca26 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:03.490408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cd3d3-0b28-4642-85b3-a590ee91ca26 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:03.490660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cd3d3-0b28-4642-85b3-a590ee91ca26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:03.528914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-933cd3d3-0b28-4642-85b3-a590ee91ca26 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:03.530095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-933cd3d3-0b28-4642-85b3-a590ee91ca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:03.557464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 240/483] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:03.603185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab6062-6cbd-4434-9844-9b9c0f180731 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:03.639382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 244/484] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:01:03.678092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cd3d3-0b28-4642-85b3-a590ee91ca26 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:03.746597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:03.756611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab6062-6cbd-4434-9844-9b9c0f180731 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:03.806601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e58a72-88ce-4589-a774-6aca049ea4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:03.807833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58a72-88ce-4589-a774-6aca049ea4de None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:03.808190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58a72-88ce-4589-a774-6aca049ea4de None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:03.808478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58a72-88ce-4589-a774-6aca049ea4de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:03.833896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e58a72-88ce-4589-a774-6aca049ea4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-54e58a72-88ce-4589-a774-6aca049ea4de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:03.841651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:03.843276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:03.844048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 241/485] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:03.850734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e58a72-88ce-4589-a774-6aca049ea4de None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:03.854738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:03.880615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cd3d3-0b28-4642-85b3-a590ee91ca26 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:03.917523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c5bf3-e273-403b-a3d6-60f39f38228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:03.918159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5bf3-e273-403b-a3d6-60f39f38228e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:03.918435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5bf3-e273-403b-a3d6-60f39f38228e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:03.918704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c5bf3-e273-403b-a3d6-60f39f38228e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:03.938968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c5bf3-e273-403b-a3d6-60f39f38228e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ce7c5bf3-e273-403b-a3d6-60f39f38228e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:03.945866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:03.946710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:03.947471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 245/486] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:03.955165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c5bf3-e273-403b-a3d6-60f39f38228e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.057929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e58a72-88ce-4589-a774-6aca049ea4de None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:04.106603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91180a-154f-41e9-926c-31293288f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:04.107583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91180a-154f-41e9-926c-31293288f38a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.108058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91180a-154f-41e9-926c-31293288f38a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.108569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91180a-154f-41e9-926c-31293288f38a None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.109538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91180a-154f-41e9-926c-31293288f38a None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:04.110060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91180a-154f-41e9-926c-31293288f38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:04.111626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91180a-154f-41e9-926c-31293288f38a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:04.121873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 242/487] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:04.127749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003aae1-42f6-439a-9df6-9bcf926e1c3b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.127951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c5bf3-e273-403b-a3d6-60f39f38228e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:04.128723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003aae1-42f6-439a-9df6-9bcf926e1c3b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.129211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003aae1-42f6-439a-9df6-9bcf926e1c3b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.131176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 243/488] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:04.132240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003aae1-42f6-439a-9df6-9bcf926e1c3b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.151142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:04.174625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c28ea8-da28-425a-b175-0a31caa55bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:04.175729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c28ea8-da28-425a-b175-0a31caa55bcf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.176297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c28ea8-da28-425a-b175-0a31caa55bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.176806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c28ea8-da28-425a-b175-0a31caa55bcf None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.177672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c28ea8-da28-425a-b175-0a31caa55bcf None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:04.178283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c28ea8-da28-425a-b175-0a31caa55bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:04.180070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c28ea8-da28-425a-b175-0a31caa55bcf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:04.191378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 246/489] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:04.192961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8020d-882e-4a2e-9ecb-b9e0d0c17c8d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.193552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8020d-882e-4a2e-9ecb-b9e0d0c17c8d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.194032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8020d-882e-4a2e-9ecb-b9e0d0c17c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.215886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-17f8020d-882e-4a2e-9ecb-b9e0d0c17c8d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:04.217465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-17f8020d-882e-4a2e-9ecb-b9e0d0c17c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:04.220345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:04.279169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003aae1-42f6-439a-9df6-9bcf926e1c3b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:04.301994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 247/490] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:04.303447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8020d-882e-4a2e-9ecb-b9e0d0c17c8d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.333757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:04.334423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.334691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.334962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.347703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:04.349343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:04.364921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:04.365850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:04.366669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:04.366669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:04.366669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:04.370966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 244/491] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:04.371808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.422380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8020d-882e-4a2e-9ecb-b9e0d0c17c8d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:04.456384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:04.459274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:04.460377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.460681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.460944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.468629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:04.473652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:04.475227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:04.492876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:04.493804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:04.494634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:04.494634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:04.494634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:04.494634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 248/492] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:04.496688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.501876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1428e-3fc0-4854-baec-8be88f3a0b3d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:04.543958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4eada-42d2-4a12-be90-3c4c61c57f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:04.544627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4eada-42d2-4a12-be90-3c4c61c57f45 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.544901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4eada-42d2-4a12-be90-3c4c61c57f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.545165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4eada-42d2-4a12-be90-3c4c61c57f45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.556276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4eada-42d2-4a12-be90-3c4c61c57f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-25c4eada-42d2-4a12-be90-3c4c61c57f45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:04.562312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:04.563233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:04.564032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 245/493] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:04.564695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8ab4e-c08b-458b-8ab1-af1d25e7f657 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.565544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8ab4e-c08b-458b-8ab1-af1d25e7f657 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.565544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8ab4e-c08b-458b-8ab1-af1d25e7f657 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.566417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 246/494] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:04.567043 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8ab4e-c08b-458b-8ab1-af1d25e7f657 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.633543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e14bd-e090-4187-bd23-12a392da8fe8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:04.677848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:04.678650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.678925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.681013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.682006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:04.682595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:04.685439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:04.701637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 249/495] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:04.702404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.707346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8ab4e-c08b-458b-8ab1-af1d25e7f657 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:04.744164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:04.756638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466551a2-a727-45b7-8e2b-311a79af8104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:04.757419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466551a2-a727-45b7-8e2b-311a79af8104 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.757705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466551a2-a727-45b7-8e2b-311a79af8104 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.757979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466551a2-a727-45b7-8e2b-311a79af8104 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.758632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466551a2-a727-45b7-8e2b-311a79af8104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:04.758936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466551a2-a727-45b7-8e2b-311a79af8104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:04.760884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466551a2-a727-45b7-8e2b-311a79af8104 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:04.764353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:04.778980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 247/496] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:04.778980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986adf5-5f1b-42eb-957f-d57e1ac2d171 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.778980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986adf5-5f1b-42eb-957f-d57e1ac2d171 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.778980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986adf5-5f1b-42eb-957f-d57e1ac2d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.795242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d986adf5-5f1b-42eb-957f-d57e1ac2d171 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:04.796596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d986adf5-5f1b-42eb-957f-d57e1ac2d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:04.827125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09412f50-75b0-4a09-9180-0a59dea36a03 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:04.867479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:04.868178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:04.868548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:04.868828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:04.869490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:04.870661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:04.871621 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 248/497] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:04.872305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986adf5-5f1b-42eb-957f-d57e1ac2d171 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.881833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:04.896612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 250/498] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:04.897562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:04.986922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986adf5-5f1b-42eb-957f-d57e1ac2d171 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.005880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.014815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b6f4f-63da-4029-be52-a5d62c60b586 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.030421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.031055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.031333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.031604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.032870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.039299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.040785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:05.054954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:05.055916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:05.056917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 249/499] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:01:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:05.058917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.063017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.063162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.063438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.063711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.071940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.074536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:05.087759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:05.089295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:05.090172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 251/500] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:05.090730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.174815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63f991-a7d0-4151-b94d-47831f2ee5c6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.197558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaab76-d802-4461-a953-5151b7619629 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.208220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb3e19-90e9-4d00-82ea-ba47578f1f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.208896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3e19-90e9-4d00-82ea-ba47578f1f45 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.209156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3e19-90e9-4d00-82ea-ba47578f1f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.209431 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3e19-90e9-4d00-82ea-ba47578f1f45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.220184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb3e19-90e9-4d00-82ea-ba47578f1f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-8edb3e19-90e9-4d00-82ea-ba47578f1f45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:05.225258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:05.226133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:05.226900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 250/501] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:05.227541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb3e19-90e9-4d00-82ea-ba47578f1f45 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.237803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.238366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.238640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.238909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.249594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.251057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.261790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:05.265449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:05.266462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:05.267265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:05.267265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:05.267265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:05.267265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 252/502] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:05.267265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.293663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.331682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb3e19-90e9-4d00-82ea-ba47578f1f45 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.365143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2b34e-f02a-4dd2-a6d7-f525e0ecc2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.365847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2b34e-f02a-4dd2-a6d7-f525e0ecc2fe None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.366127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2b34e-f02a-4dd2-a6d7-f525e0ecc2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.366394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2b34e-f02a-4dd2-a6d7-f525e0ecc2fe None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.367105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2b34e-f02a-4dd2-a6d7-f525e0ecc2fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:05.367405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2b34e-f02a-4dd2-a6d7-f525e0ecc2fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.368427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4dbaa-a901-4319-9760-ee088603e6e8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.368861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2b34e-f02a-4dd2-a6d7-f525e0ecc2fe None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.406771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 251/503] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:05.408028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d0bbd-7d2d-4e85-bf63-5aef0e69d63b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.408314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d0bbd-7d2d-4e85-bf63-5aef0e69d63b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.408616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d0bbd-7d2d-4e85-bf63-5aef0e69d63b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.409823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 252/504] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:05.410504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d0bbd-7d2d-4e85-bf63-5aef0e69d63b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.419177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.420210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.420520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.420799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.421566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:05.421876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.423317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.461537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 253/505] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:05.462419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.508190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.516359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d0bbd-7d2d-4e85-bf63-5aef0e69d63b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.545964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.551154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.551756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.552035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.552365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.553060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:05.553388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.554752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.564623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 253/506] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:05.565381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70cbf2-3920-49f1-86e4-9413324b1c77 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.565500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.600119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b07286-9fb9-4d6a-9006-3cf9f9c2cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.600754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b07286-9fb9-4d6a-9006-3cf9f9c2cc75 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.601028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b07286-9fb9-4d6a-9006-3cf9f9c2cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.601298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b07286-9fb9-4d6a-9006-3cf9f9c2cc75 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.601940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b07286-9fb9-4d6a-9006-3cf9f9c2cc75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:05.602236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b07286-9fb9-4d6a-9006-3cf9f9c2cc75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.603615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b07286-9fb9-4d6a-9006-3cf9f9c2cc75 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.615248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 254/507] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:05.616479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550b7a0-caff-48ba-b64f-5d8cf71d0e54 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.616769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550b7a0-caff-48ba-b64f-5d8cf71d0e54 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.617045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550b7a0-caff-48ba-b64f-5d8cf71d0e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.638751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6550b7a0-caff-48ba-b64f-5d8cf71d0e54 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:05.639948 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6550b7a0-caff-48ba-b64f-5d8cf71d0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:05.673532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9de68-f400-41b2-a9ac-63f7f9743d8c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.709258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d833a5-1731-4b87-8848-dc36e4d8031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.709981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d833a5-1731-4b87-8848-dc36e4d8031e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.710264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d833a5-1731-4b87-8848-dc36e4d8031e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.710538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d833a5-1731-4b87-8848-dc36e4d8031e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.714274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 255/508] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:05.715186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550b7a0-caff-48ba-b64f-5d8cf71d0e54 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.739896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.782932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d833a5-1731-4b87-8848-dc36e4d8031e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.784181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d833a5-1731-4b87-8848-dc36e4d8031e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.786778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.817007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550b7a0-caff-48ba-b64f-5d8cf71d0e54 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.850580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2245d-acac-4576-b767-52da47c6fd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:05.851173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2245d-acac-4576-b767-52da47c6fd9c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:05.851448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2245d-acac-4576-b767-52da47c6fd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:05.851716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2245d-acac-4576-b767-52da47c6fd9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:05.884330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 254/509] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:05.885194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d833a5-1731-4b87-8848-dc36e4d8031e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:05.924268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2245d-acac-4576-b767-52da47c6fd9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:05.925827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2245d-acac-4576-b767-52da47c6fd9c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:05.942104 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:05.986396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d833a5-1731-4b87-8848-dc36e4d8031e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:05.994163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.021619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.022235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.022513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.022784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.026393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 256/510] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:06.027173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2245d-acac-4576-b767-52da47c6fd9c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.035690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.038433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:06.058261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:06.059438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:06.060753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:06.060753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:06.060753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:06.060753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 255/511] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:06.061193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.136453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2245d-acac-4576-b767-52da47c6fd9c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.170295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8bb1d-01e6-4d47-b330-44e8478ba8dd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.171631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.172561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.172836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.179760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.180656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.181233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:06.195215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:06.196312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 257/512] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:06.197873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.208602 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.209256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.209534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.209805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.220028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.221505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.229640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:06.235964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:06.237100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:06.237948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:06.237948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:06.237948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:06.237948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 256/513] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:06.237948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.298402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4af4bd-b9ab-4cf9-98e6-38a51a7d6d97 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.332473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.333095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.333371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.333643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.339186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14087692-fc18-4ea5-a356-dd4c753807bc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.343979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.345458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:06.359693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:06.360795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:06.361841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:06.361841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:06.361841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:06.361841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 258/514] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:06.361841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.375566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.376276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.376579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.376855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.388189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.391110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:06.412359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:06.413558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:06.414584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:06.414584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:06.414584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:06.414584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 257/515] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:06.414584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.430903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.462457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.481324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ec8f3-df3f-4b2e-92d6-374d4042ddf4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.515822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.516420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.516725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.516816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fb35d-498b-4d3f-962e-95d3ea43a8b5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.517230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.517620 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:06.517918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.519293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.530699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 259/516] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:06.531536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.551241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.551774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.552050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.552321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.553074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:06.553378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.554826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.572610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 258/517] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:06.574542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.633087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b0620-294c-4849-a1df-c083c07fa8c2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.643078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.666885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.667466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.667740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.668030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.668697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:06.668996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.670365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.675052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b019dc-eeff-4a05-bc52-4889afd9d91a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.681284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 260/518] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:06.682079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.687877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.711467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.712070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.712347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.712654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.713282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:06.713735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.715095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.725774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 259/519] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:06.726547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.784680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708986a-8b79-4e59-aef4-41fbcbbabc5a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.818807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.819333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.819605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.819872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.820599 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:06.820908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.822266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.829233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f8345-9a76-4122-8f9d-74593cea49c6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.833757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 261/520] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:06.834605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.859603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.865699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.866322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.866598 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.866868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.874849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:06.876285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:06.887132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:06.888204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:06.889322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 260/521] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:06.889972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:06.900059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:06.940779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bdac1-91d8-4c28-a1cf-3b01ac3fed51 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:06.973155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b9526-8815-4bfb-817e-ad292d63d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:06.973782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b9526-8815-4bfb-817e-ad292d63d53a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:06.974054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b9526-8815-4bfb-817e-ad292d63d53a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:06.974317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b9526-8815-4bfb-817e-ad292d63d53a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:06.987755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380d5f5-ee7a-41d4-bf0d-0e1565a54c90 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.020174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ad37f-07b6-4fc1-b8de-ab0d55000a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.020813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ad37f-07b6-4fc1-b8de-ab0d55000a49 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.021087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ad37f-07b6-4fc1-b8de-ab0d55000a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.021357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ad37f-07b6-4fc1-b8de-ab0d55000a49 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.021972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ad37f-07b6-4fc1-b8de-ab0d55000a49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.023381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ad37f-07b6-4fc1-b8de-ab0d55000a49 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.046292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 261/522] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:01:06 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:07.047477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ad37f-07b6-4fc1-b8de-ab0d55000a49 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.066008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b9526-8815-4bfb-817e-ad292d63d53a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.067988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b9526-8815-4bfb-817e-ad292d63d53a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.108204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:07.139476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:07.176781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ad37f-07b6-4fc1-b8de-ab0d55000a49 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.185305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 262/523] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:07.186274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b9526-8815-4bfb-817e-ad292d63d53a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.218876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.219605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.219948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.220282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.228671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.230785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:07.245911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:07.247020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:07.248075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:07.248075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:07.248075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:07.248075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:07.248075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:07.248075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:07.248075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:07.248075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:07.248733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 262/524] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:07.249829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.306981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b9526-8815-4bfb-817e-ad292d63d53a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.343404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.344141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.344413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.344827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.351750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.353148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62512f4-0596-42cb-9b32-eccc35c6dac0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.353286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.361242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:07.366302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:07.367312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:07.368429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:07.368429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:07.368429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:07.368429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:07.368429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:07.368429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:07.368429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:07.368429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:07.372212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 263/525] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:07.373073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.401359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.402239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.402583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.402910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.403697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:07.404065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.405342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:07.405596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.426183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 263/526] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:07.431246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.506517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38893e-bc70-4eb3-9fb2-a881b00d184d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.542709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.543338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.543616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.543890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.553093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f3397-4287-4bed-93fa-5bcfc45de29f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.554435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.555907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:07.570203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:07.571190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:07.572110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:07.572110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:07.572110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:07.572110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 264/527] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:07.572110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.599569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.600613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.601114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.601654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.602352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:07.602666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.604356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.625253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 264/528] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:07.626005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.655013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:07.665023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:07.690562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8947-d812-49fe-9931-d97689d92aaa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.727596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.728234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.728545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.728819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.730341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:07.730754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.732188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.735554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395a699-7253-4638-bd15-652007c9c02d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.756474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 265/529] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:07.757350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.772016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9429d-81a8-4f09-9772-09d9e7787d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.772803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9429d-81a8-4f09-9772-09d9e7787d15 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.773185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9429d-81a8-4f09-9772-09d9e7787d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.773558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9429d-81a8-4f09-9772-09d9e7787d15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.849493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9429d-81a8-4f09-9772-09d9e7787d15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.851175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9429d-81a8-4f09-9772-09d9e7787d15 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.866187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b1a66-c38e-4893-b891-bb6c4b03d30b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:07.879753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:07.906710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:07.907473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:07.907817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:07.908164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:07.908914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:07.909306 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:07.910748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:07.913582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:07.924122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 266/530] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:07.924997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:07.971892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 265/531] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:07.971892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9429d-81a8-4f09-9772-09d9e7787d15 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:08.042685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282306f-dfe9-4d46-afaa-ab8bc3d538de None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:08.081046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9429d-81a8-4f09-9772-09d9e7787d15 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:08.081238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:08.081816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:08.082145 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:08.082478 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:08.083205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=48410d57fe0d489cbb07cfca58ca7b30) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:08.083532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:08.085000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:08.100476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 267/532] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/role_assignments?user.id=48410d57fe0d489cbb07cfca58ca7b30&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:08.100476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:08.121154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:08.139961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:08.140627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:08.140978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:08.141599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:08.143409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=92a88817f3a4402290e9358826a63904) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:08.143799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:08.145372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:08.156660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 266/533] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:07 2022] GET /identity/v3/role_assignments?user.id=92a88817f3a4402290e9358826a63904&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:08.157677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:08.200537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:08.296159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b84630-9a7d-40f4-a320-c3655c52c625 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:08.346481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60c130-2a39-45be-a146-00217d179f79 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:08.364630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:08.365325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:08.365603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:08.365870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:08.378049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:08.379521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:08.408612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:08.409506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:08.410341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:08.410341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:08.410341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:08.410341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:08.410341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:08.410341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:08.410341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:08.410341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:08.413491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 268/534] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:08.420213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:08.425072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:08.425715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:08.426235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:08.426511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:08.448630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:08.450563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:08.473973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:08.474913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:08.475740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:08.475740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:08.475740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:08.477546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 267/535] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:08.483778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:08.534148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:08.605900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cedff3-f10f-42d1-b6d4-57180850f530 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:08.614332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36698c-a50a-44e9-9ad6-5a17bad41801 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:08.646827 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:08.647757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:08.648236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:08.648756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:08.649706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:08.650449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:08.652923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:08.656898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:08.672650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 268/536] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:08.674935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:08.697340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:08.698076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:08.698349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:08.698610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:08.699353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:08.699658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:08.701122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:08.722445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 269/537] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:08.730235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:08.874264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:08.878881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e703c6-a694-4b78-8d2e-5ba9f6b1b6c8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:08.889584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac0f8a-8b87-4605-ba01-9256e0628808 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:08.943304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:08.944494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:08.944838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:08.945113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:08.946389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:08.947116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:08.947393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:08.947871 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:08.948927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=00be9b6d551a4ba39718a44101ecf9ca) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:08.949239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:08.950694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:08.953060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:08.954747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:08.965673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 270/538] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:08 2022] GET /identity/v3/role_assignments?user.id=00be9b6d551a4ba39718a44101ecf9ca&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:08.975088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:08.975985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:08.976832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:08.976832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:08.976832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:08.976832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:08.976832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:08.976832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:08.976832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:08.976832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:08.979991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 269/539] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:08.985620 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:09.079572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:09.166211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a541dee-581a-47a1-aa10-5351bf0f5fdd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:09.203510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c42dd-4a38-452b-8e28-7fb5921109d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:09.204038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c42dd-4a38-452b-8e28-7fb5921109d5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:09.204315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c42dd-4a38-452b-8e28-7fb5921109d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:09.204637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c42dd-4a38-452b-8e28-7fb5921109d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:09.205324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c42dd-4a38-452b-8e28-7fb5921109d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:09.205626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c42dd-4a38-452b-8e28-7fb5921109d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:09.207045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c42dd-4a38-452b-8e28-7fb5921109d5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:09.219852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 271/540] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:09.225362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:09.364933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772c2f1-bd2f-4d70-bc47-78c6c8fc7f32 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:09.426108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708ad12-a485-4917-8f06-0b964edbe027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:09.426730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708ad12-a485-4917-8f06-0b964edbe027 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:09.427007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708ad12-a485-4917-8f06-0b964edbe027 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:09.427276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708ad12-a485-4917-8f06-0b964edbe027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:09.428011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708ad12-a485-4917-8f06-0b964edbe027 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=31183ac0fdd642e6a1a7578899aa9fdf) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:09.428313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708ad12-a485-4917-8f06-0b964edbe027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:09.429851 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708ad12-a485-4917-8f06-0b964edbe027 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:09.437435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 270/541] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:09 2022] GET /identity/v3/role_assignments?user.id=31183ac0fdd642e6a1a7578899aa9fdf&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:09.462203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb126-29ca-4dd3-a64c-8191714a1238 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:09.462802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb126-29ca-4dd3-a64c-8191714a1238 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:09.463835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bb126-29ca-4dd3-a64c-8191714a1238 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:09.467390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 272/542] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:09.472421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe70b1-0ec7-4d1d-a38c-02886265119e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:09.472747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe70b1-0ec7-4d1d-a38c-02886265119e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:09.473000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe70b1-0ec7-4d1d-a38c-02886265119e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:09.500145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8cfe70b1-0ec7-4d1d-a38c-02886265119e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:09.501754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8cfe70b1-0ec7-4d1d-a38c-02886265119e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:09.525337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:09.602963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 271/543] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:09.642527 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bb126-29ca-4dd3-a64c-8191714a1238 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:09.769203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bb126-29ca-4dd3-a64c-8191714a1238 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:09.772114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:09.815482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d53b9-fb65-444b-92b3-41991c759a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:09.816120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d53b9-fb65-444b-92b3-41991c759a2c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:09.816398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d53b9-fb65-444b-92b3-41991c759a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:09.816694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d53b9-fb65-444b-92b3-41991c759a2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:09.833578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d53b9-fb65-444b-92b3-41991c759a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f50d53b9-fb65-444b-92b3-41991c759a2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:09.838341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:09.839835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:09.840628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 273/544] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:09.846272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe70b1-0ec7-4d1d-a38c-02886265119e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:09.965661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe70b1-0ec7-4d1d-a38c-02886265119e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:10.007859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:10.008504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.008788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.009058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.009724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:10.010027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:10.012202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:10.025402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 272/545] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:10.031708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d53b9-fb65-444b-92b3-41991c759a2c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:10.137547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d53b9-fb65-444b-92b3-41991c759a2c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:10.178065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aef88b-98d3-458b-ac18-33d2ae1fd954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:10.179061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aef88b-98d3-458b-ac18-33d2ae1fd954 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.179546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aef88b-98d3-458b-ac18-33d2ae1fd954 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.180006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aef88b-98d3-458b-ac18-33d2ae1fd954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.200944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aef88b-98d3-458b-ac18-33d2ae1fd954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:10.202747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aef88b-98d3-458b-ac18-33d2ae1fd954 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-52aef88b-98d3-458b-ac18-33d2ae1fd954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:10.222040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:10.222951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:10.223767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:10.223767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:10.223767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:10.227769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 274/546] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:10.229715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:10.259929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:10.381915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3689ca-c0e7-495b-8315-60453d031b38 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:10.421222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f78dd5-fbee-45ae-98fe-a43f8a908603 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.421558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f78dd5-fbee-45ae-98fe-a43f8a908603 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.421769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f78dd5-fbee-45ae-98fe-a43f8a908603 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.423114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 275/547] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:10.428296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525076b-bd62-408d-88d6-fa8d1a65aad8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.428609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525076b-bd62-408d-88d6-fa8d1a65aad8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.428893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525076b-bd62-408d-88d6-fa8d1a65aad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.443254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:10.443863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.444142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.444412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.445097 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:10.445396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:10.446765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:10.454100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3525076b-bd62-408d-88d6-fa8d1a65aad8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:10.456197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3525076b-bd62-408d-88d6-fa8d1a65aad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:10.460593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 273/548] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:10.466171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:10.475658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:10.576812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:10.586120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 276/549] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:01:10.609140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4014132-669a-43e8-93d0-58cf4819cbc6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:10.621085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525076b-bd62-408d-88d6-fa8d1a65aad8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:10.689837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:10.690477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.690848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.691133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.705735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:10.708342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:10.730925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:10.732490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:10.734075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 274/550] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:10.737701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:10.852945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21329ad-71be-4e73-94a2-b47bee23439b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:10.882391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525076b-bd62-408d-88d6-fa8d1a65aad8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:10.892401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2b118-b514-4eea-a54e-c87f926fb379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:10.893185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2b118-b514-4eea-a54e-c87f926fb379 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.893531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2b118-b514-4eea-a54e-c87f926fb379 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.893870 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2b118-b514-4eea-a54e-c87f926fb379 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.894733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2b118-b514-4eea-a54e-c87f926fb379 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:10.895110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2b118-b514-4eea-a54e-c87f926fb379 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:10.896669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2b118-b514-4eea-a54e-c87f926fb379 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:10.899869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:10.923403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 275/551] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Aug 9 19:01:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:10.924863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092331ac-d249-48cd-b9c7-43321a0578df None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.925211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092331ac-d249-48cd-b9c7-43321a0578df None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.925551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092331ac-d249-48cd-b9c7-43321a0578df None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.926809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 276/552] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:10.932412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092331ac-d249-48cd-b9c7-43321a0578df None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:10.936260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ca46-4a92-4d1d-8e89-486ff577f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:10.937002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ca46-4a92-4d1d-8e89-486ff577f922 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.937349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ca46-4a92-4d1d-8e89-486ff577f922 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.937690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ca46-4a92-4d1d-8e89-486ff577f922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.952646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ca46-4a92-4d1d-8e89-486ff577f922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f815ca46-4a92-4d1d-8e89-486ff577f922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:10.959093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:10.960047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:10.961653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 277/553] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:10.962844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2520e-9df0-46e1-914e-a7cdeef38f35 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:10.962844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2520e-9df0-46e1-914e-a7cdeef38f35 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:10.962844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2520e-9df0-46e1-914e-a7cdeef38f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:10.980805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:10.989610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-22b2520e-9df0-46e1-914e-a7cdeef38f35 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:10.991736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-22b2520e-9df0-46e1-914e-a7cdeef38f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:11.072460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 278/554] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:11.072460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2520e-9df0-46e1-914e-a7cdeef38f35 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.090092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092331ac-d249-48cd-b9c7-43321a0578df None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:11.138580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6de4f6-a5b7-4af4-b6bc-1f63925a64fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:11.139567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6de4f6-a5b7-4af4-b6bc-1f63925a64fe None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.140627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6de4f6-a5b7-4af4-b6bc-1f63925a64fe None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.141149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6de4f6-a5b7-4af4-b6bc-1f63925a64fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.205745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2520e-9df0-46e1-914e-a7cdeef38f35 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:11.234911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:11.243729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:11.244167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.244558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.244829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.245499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:11.245800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:11.247180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:11.247514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6de4f6-a5b7-4af4-b6bc-1f63925a64fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:11.249355 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6de4f6-a5b7-4af4-b6bc-1f63925a64fe None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:11.249484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:11.263677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 279/555] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:11.264745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.377094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989086c3-f775-4ca7-99e9-b3df5842359c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:11.395491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 277/556] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:11.396336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6de4f6-a5b7-4af4-b6bc-1f63925a64fe None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.421673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c38ea-ad3e-4683-832a-86c51045a081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:11.422397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c38ea-ad3e-4683-832a-86c51045a081 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.422640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c38ea-ad3e-4683-832a-86c51045a081 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.422915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c38ea-ad3e-4683-832a-86c51045a081 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.436009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c38ea-ad3e-4683-832a-86c51045a081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-579c38ea-ad3e-4683-832a-86c51045a081 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:11.440712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:11.441629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:11.442413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 280/557] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:11.443062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb6d2-6b09-4f98-ab62-52b52a995856 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.443625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb6d2-6b09-4f98-ab62-52b52a995856 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.443625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb6d2-6b09-4f98-ab62-52b52a995856 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.448900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 281/558] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:11.448900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb6d2-6b09-4f98-ab62-52b52a995856 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.505251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:11.513077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6de4f6-a5b7-4af4-b6bc-1f63925a64fe None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:11.513964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:11.551289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:11.551915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.552280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.552487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.563700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb6d2-6b09-4f98-ab62-52b52a995856 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:11.564371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:11.565825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:11.580923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:11.581882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:11.582790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:11.582790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:11.582790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:11.582790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 278/559] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:11.583457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.604731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd496b-6e1e-4226-845a-0cbe0e3dfbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:11.605380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd496b-6e1e-4226-845a-0cbe0e3dfbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.605639 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd496b-6e1e-4226-845a-0cbe0e3dfbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.605914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd496b-6e1e-4226-845a-0cbe0e3dfbc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.617042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd496b-6e1e-4226-845a-0cbe0e3dfbc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:11.618489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd496b-6e1e-4226-845a-0cbe0e3dfbc8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7bdd496b-6e1e-4226-845a-0cbe0e3dfbc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:11.633702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:11.634703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:11.635848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:11.635848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:11.635848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:11.635848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 282/560] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:11.636291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497545bf-ca8c-4802-9922-af47bc476bce None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.636291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497545bf-ca8c-4802-9922-af47bc476bce None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.636912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497545bf-ca8c-4802-9922-af47bc476bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.655233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-497545bf-ca8c-4802-9922-af47bc476bce None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:11.657122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-497545bf-ca8c-4802-9922-af47bc476bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:11.698991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef8710-eb7d-4262-9343-3e980f191254 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:11.740492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 283/561] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:11.740723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:11.740837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497545bf-ca8c-4802-9922-af47bc476bce None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.740937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.740937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.740937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.740937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:11.740937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:11.740937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:11.757704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 279/562] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:11.758853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.770221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:11.789170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:11.853701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497545bf-ca8c-4802-9922-af47bc476bce None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:11.863986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08729d0b-fa91-42f3-9d31-88622ead2fc3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:11.891659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:11.892477 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.892586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.892826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.893459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:11.893757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:11.895113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:11.898322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:11.898926 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:11.899202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:11.899472 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:11.900105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:11.900402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:11.902556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:11.908124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 284/563] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:11.909185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.923526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 280/564] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:11.924219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:11.998412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.018304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.022124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8dfac-3edf-412d-beb7-a9c68e0c04b3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.035217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c30d2-3acf-46dc-b363-b1ee3c4e8af5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.060692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.061320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.061595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.061867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.073196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.074186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a57cb-43e9-468c-b9dc-5467e321c7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.074646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.074823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a57cb-43e9-468c-b9dc-5467e321c7ab None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.075103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a57cb-43e9-468c-b9dc-5467e321c7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.075370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a57cb-43e9-468c-b9dc-5467e321c7ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.085340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a57cb-43e9-468c-b9dc-5467e321c7ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:12.089320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:12.090217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:12.091091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:12.091091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:12.091091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:12.091091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 285/565] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a98a57cb-43e9-468c-b9dc-5467e321c7ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:12.091308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:12.092127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:12.093736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:12.095056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.095194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 281/566] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:12.095194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a57cb-43e9-468c-b9dc-5467e321c7ab None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.198133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a958fe-0d0c-4b3e-911e-ae8f2adb0d95 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.200966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a57cb-43e9-468c-b9dc-5467e321c7ab None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.231425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe831e9c-a6e1-48d5-a1ca-64733718cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.232040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831e9c-a6e1-48d5-a1ca-64733718cfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.232313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831e9c-a6e1-48d5-a1ca-64733718cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.232621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831e9c-a6e1-48d5-a1ca-64733718cfe9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.235568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.240093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe831e9c-a6e1-48d5-a1ca-64733718cfe9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.241564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe831e9c-a6e1-48d5-a1ca-64733718cfe9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.244257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.245758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.246183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.246522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fe831e9c-a6e1-48d5-a1ca-64733718cfe9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:12.252997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:12.254068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:12.254977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:12.255492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.255567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 286/567] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:12.255641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.255709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e54849-42da-4a34-99a0-8f1dbcd42ed7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.255709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e54849-42da-4a34-99a0-8f1dbcd42ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.255709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e54849-42da-4a34-99a0-8f1dbcd42ed7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.257019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 287/568] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:12.257534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e54849-42da-4a34-99a0-8f1dbcd42ed7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:12.267249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:12.268144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 282/569] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:12.274264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.277016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.362818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e54849-42da-4a34-99a0-8f1dbcd42ed7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.367850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe58131-e9c4-43ae-8e90-a3ec28ba0a0b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.396342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f50731-f075-4b6f-8263-615a13f131dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.396976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f50731-f075-4b6f-8263-615a13f131dc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.397256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f50731-f075-4b6f-8263-615a13f131dc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.397527 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f50731-f075-4b6f-8263-615a13f131dc None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.398186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f50731-f075-4b6f-8263-615a13f131dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:12.398486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f50731-f075-4b6f-8263-615a13f131dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.399845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f50731-f075-4b6f-8263-615a13f131dc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.401841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.402437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.402713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.402980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.403607 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:12.403905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.405260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.409028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 288/570] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:12.410206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8757ad-8f7e-4e9f-b410-0b0458c09a39 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.410490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8757ad-8f7e-4e9f-b410-0b0458c09a39 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.410762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8757ad-8f7e-4e9f-b410-0b0458c09a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.417556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 283/571] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:12.418475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.428993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2c8757ad-8f7e-4e9f-b410-0b0458c09a39 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:12.430202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2c8757ad-8f7e-4e9f-b410-0b0458c09a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:12.467463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.496720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 289/572] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:12.497616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8757ad-8f7e-4e9f-b410-0b0458c09a39 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.512037 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.520811 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c2d66-777c-4272-b9c2-76226e4c3349 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.555595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.556138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.556411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.556707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.557986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:12.558310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.559721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.577096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 284/573] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:12.577893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.611635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8757ad-8f7e-4e9f-b410-0b0458c09a39 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.645590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.646189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.646464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.646733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.647413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:12.647713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.649167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.660272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 290/574] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:12.661079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.675480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfd6a1-d402-43f7-938e-f94e394800e7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.683371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.712035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.712695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.712974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.713330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.724164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.724719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.725740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:12.741619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:12.742578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:12.743451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:12.743451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:12.743451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:12.743451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 285/575] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:12.743451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.766182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e7f3c-72d0-471e-a5a3-4bb4e63a80c9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.799079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.799693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.799968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.800238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.808097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.809493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:12.820692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:12.821725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 291/576] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:12.823186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.848972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a8f10-08e5-4314-904f-b1019e41b634 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.881900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83017233-d33f-428c-897a-4a855acdec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.882516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83017233-d33f-428c-897a-4a855acdec67 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.882781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83017233-d33f-428c-897a-4a855acdec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.883048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83017233-d33f-428c-897a-4a855acdec67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.893758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83017233-d33f-428c-897a-4a855acdec67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-83017233-d33f-428c-897a-4a855acdec67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:12.898757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:12.899680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 286/577] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:12.901859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83017233-d33f-428c-897a-4a855acdec67 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:12.922056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.924142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910c382-02da-42f9-ba57-54254497af22 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:12.948117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:12.959177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:12.959798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:12.960074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:12.960342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:12.971001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:12.972469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:12.986302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:12.987255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:12.988142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:12.988142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:12.988142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:12.988142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 292/578] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:12.988142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.007697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83017233-d33f-428c-897a-4a855acdec67 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.042121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.042738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.043010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.043282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.043996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=141d2c2f92c842feb6155aae1e9edfd2) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:13.044300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.045737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.051076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 287/579] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/role_assignments?user.id=141d2c2f92c842feb6155aae1e9edfd2&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:13.051920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.112110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d3130-7564-4d63-a801-13893b2e4939 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.151828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:13.155309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.156023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.156371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.156961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.157469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:13.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.159343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.178922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 293/580] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:13.179116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1a4bd-3c81-4563-bb15-d204ae711b0a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.179228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.222498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:13.223571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.224247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.224629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.224965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.225759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:13.226129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.227748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.249907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 288/581] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:13.251371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.304581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88948f6-aa62-490a-b4df-73e39a80b44d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.344093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.344729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.345007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.345278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.345941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:13.346241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.347606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.357800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 294/582] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:13.358925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.375366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762a183-43f8-4770-85cc-fc9feb18262a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.420188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.420824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.421103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.421377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.422038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:13.422338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.423705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.432533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:13.442024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 289/583] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:13.443251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.477160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:13.481881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba31b7-1ec9-49ab-b7d3-6e4e1dab1ab0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.520481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.521344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.521684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.522020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.530218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.531657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:13.544964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:13.545934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 295/584] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:13.546866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.558159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f894-32e0-48e9-b1c3-edce75670ad8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.593013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.593649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.593929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.594203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.605712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.607158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:13.622160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:13.623317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:13.624197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:13.624197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:13.624197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:13.624197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 290/585] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:13.624415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.662916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afccd2-9082-4233-8743-2072320727a4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.684333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:13.698653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.699459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.699864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.700144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.711635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.713170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:13.727803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:13.729351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:13.730210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:13.730210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:13.730210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:13.730210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 296/586] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:13.730862 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.743023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df8fa7-4997-4496-bea1-cb7a80d4c0a7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.744053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:13.782759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.783734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.784203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.784501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.791962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.793636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:13.804783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:13.805684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:13.806546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 291/587] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:13.807666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.847290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78556da2-eae6-41a4-a087-58df0e7f3220 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.881651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.882255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.882534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.882807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.883527 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:13.883836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.885447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.903552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 297/588] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:13.904360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.921176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a15d4-5b64-45c5-88ff-6ac31b30f1ec None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:13.934127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:13.956648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:13.957258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:13.957554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:13.957816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:13.958457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:13.958756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:13.960109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:13.972631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 292/589] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:13.973429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:13.986775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:14.022452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba749961-07dd-4010-aaa7-cd06fdaf5c93 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.060738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.061348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.061625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.061896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.062539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:14.062841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.064293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.080601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 298/590] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:14.081512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.087460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e415-31d8-4bb4-a244-bc07d753ceed None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.130019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.130911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.131193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.131467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.132336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:14.132964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.134386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.149097 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 293/591] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:14.149959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.190903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:14.196521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe3749-41c6-4cbf-b7a8-56ff74ca9196 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.226360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:14.235370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.236004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.236274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.236573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.247614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.249449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:14.265022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:14.265987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:14.266841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:14.266841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:14.266841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:14.266841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 299/592] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:14.267069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.274618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3461dca-bf0d-4e5e-9a41-c189fec43a02 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.312312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.313023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.313405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.313704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.321537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.323946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:14.336521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:14.337387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:14.338907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 294/593] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:14.339400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.394448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f023bb-2875-47f6-bedb-fd5576b7cc64 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.430235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.430849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.431124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.431388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.439232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.440691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.442997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:14.452817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:14.453781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 300/594] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:14.454716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.465992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4925f4-13b9-44c0-9031-5bd8fa169079 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.502441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:14.504340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d903a-72da-46e9-9667-1bdc9bbf8eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.505147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d903a-72da-46e9-9667-1bdc9bbf8eda None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.505494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d903a-72da-46e9-9667-1bdc9bbf8eda None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.505811 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d903a-72da-46e9-9667-1bdc9bbf8eda None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/48410d57fe0d489cbb07cfca58ca7b30/roles/72882169372f4db5bd4bf8c9c399b812` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.523511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d903a-72da-46e9-9667-1bdc9bbf8eda None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=48410d57fe0d489cbb07cfca58ca7b30, role_id=72882169372f4db5bd4bf8c9c399b812)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.525100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d903a-72da-46e9-9667-1bdc9bbf8eda None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.548494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 295/595] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:14 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/48410d57fe0d489cbb07cfca58ca7b30/roles/72882169372f4db5bd4bf8c9c399b812 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:14.548494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d903a-72da-46e9-9667-1bdc9bbf8eda None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.576052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d5567-7c66-4c17-af0c-f89f3a12f9a2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.616289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.617040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.617324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.617941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.619036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:14.619345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.621821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.643891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 301/596] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:14.645177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.698327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d903a-72da-46e9-9667-1bdc9bbf8eda None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.738721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.739628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.739910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.740182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.740909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:14.741216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.742626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.756188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 296/597] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:14.757068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.757702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:14.760153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:14.764939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6ab1-7f91-49c8-bc8f-b5c0c8d1f4a2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.809525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.810485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.810712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.810983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.811759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:14.812082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.813576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.832802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 302/598] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Aug 9 19:01:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:14.833629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.876167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4b46d-f0f1-4900-99aa-37f6859144cf None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:14.915774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:14.916395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:14.917412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:14.917854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:14.925234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:14.927886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:14.940249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:14.941201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 297/599] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:14.942693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:14.971598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d0305-d02a-4f24-b903-a20e62992fed None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.013418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:15.023622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f090e4-98ff-45e3-9ff3-6ef224d65d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.025201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f090e4-98ff-45e3-9ff3-6ef224d65d41 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.025483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f090e4-98ff-45e3-9ff3-6ef224d65d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.026334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f090e4-98ff-45e3-9ff3-6ef224d65d41 None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/92a88817f3a4402290e9358826a63904/roles/72882169372f4db5bd4bf8c9c399b812` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.044294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f090e4-98ff-45e3-9ff3-6ef224d65d41 None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=92a88817f3a4402290e9358826a63904, role_id=72882169372f4db5bd4bf8c9c399b812)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:15.045780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f090e4-98ff-45e3-9ff3-6ef224d65d41 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:15.066090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 303/600] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:14 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/92a88817f3a4402290e9358826a63904/roles/72882169372f4db5bd4bf8c9c399b812 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:15.066090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f090e4-98ff-45e3-9ff3-6ef224d65d41 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:15.077320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ef5d4-409e-445b-b198-251965956f80 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.077939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:15.130878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685dbb05-499c-437a-bca7-e043e4e71d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.131503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dbb05-499c-437a-bca7-e043e4e71d4a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.131784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dbb05-499c-437a-bca7-e043e4e71d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.132055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dbb05-499c-437a-bca7-e043e4e71d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.145877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685dbb05-499c-437a-bca7-e043e4e71d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:15.147372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685dbb05-499c-437a-bca7-e043e4e71d4a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-685dbb05-499c-437a-bca7-e043e4e71d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:15.164005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:15.165075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:15.167391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:15.167391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:15.167391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:15.167391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 298/601] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:15.167998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ae20-24b6-4a7c-8589-b5c2d28b5461 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.168177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ae20-24b6-4a7c-8589-b5c2d28b5461 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.168459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ae20-24b6-4a7c-8589-b5c2d28b5461 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.170359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 299/602] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:15.171372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7ae20-24b6-4a7c-8589-b5c2d28b5461 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:15.225043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f090e4-98ff-45e3-9ff3-6ef224d65d41 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.270580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8cd6e-7f29-4ba7-a496-ffd5d1fc2665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.270786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8cd6e-7f29-4ba7-a496-ffd5d1fc2665 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.270993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8cd6e-7f29-4ba7-a496-ffd5d1fc2665 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.273523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8cd6e-7f29-4ba7-a496-ffd5d1fc2665 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.273523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8cd6e-7f29-4ba7-a496-ffd5d1fc2665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:15.273523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8cd6e-7f29-4ba7-a496-ffd5d1fc2665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:15.273747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8cd6e-7f29-4ba7-a496-ffd5d1fc2665 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:15.290626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 304/603] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:15.292643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4be4f-d557-4cb8-8bff-1ebb2993e9b9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.292938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4be4f-d557-4cb8-8bff-1ebb2993e9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.293183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4be4f-d557-4cb8-8bff-1ebb2993e9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.305573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7ae20-24b6-4a7c-8589-b5c2d28b5461 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.317050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-08d4be4f-d557-4cb8-8bff-1ebb2993e9b9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:15.318806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-08d4be4f-d557-4cb8-8bff-1ebb2993e9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:15.321939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:15.344604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.345300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.345642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.345974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.346742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:15.347127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:15.348612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:15.362332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 300/604] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:15.363504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:15.406474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:15.412484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 305/605] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:15.413367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4be4f-d557-4cb8-8bff-1ebb2993e9b9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:15.513313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbdcb4-86db-4674-8c61-8a1073e91d0d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.548490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4be4f-d557-4cb8-8bff-1ebb2993e9b9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.549863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f02198-3557-4ba7-ad3e-e39ed19f8254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.550592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f02198-3557-4ba7-ad3e-e39ed19f8254 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.550933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f02198-3557-4ba7-ad3e-e39ed19f8254 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.551272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f02198-3557-4ba7-ad3e-e39ed19f8254 None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/00be9b6d551a4ba39718a44101ecf9ca/roles/72882169372f4db5bd4bf8c9c399b812` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.567892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f02198-3557-4ba7-ad3e-e39ed19f8254 None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=00be9b6d551a4ba39718a44101ecf9ca, role_id=72882169372f4db5bd4bf8c9c399b812)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:15.569510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f02198-3557-4ba7-ad3e-e39ed19f8254 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:15.584276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 301/606] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:15 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/00be9b6d551a4ba39718a44101ecf9ca/roles/72882169372f4db5bd4bf8c9c399b812 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:15.585524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f02198-3557-4ba7-ad3e-e39ed19f8254 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:15.595023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.595997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.596500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.596960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.617705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:15.619601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:15.620661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:15.645130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:15.646266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:15.647360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:15.647360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:15.647360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:15.647360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:15.647360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:15.647360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:15.647360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:15.647360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:15.651711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 306/607] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:15.657518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:15.709085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f02198-3557-4ba7-ad3e-e39ed19f8254 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.750202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf6942-3bcb-4fc2-8df0-cb11f0be9b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.750850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf6942-3bcb-4fc2-8df0-cb11f0be9b44 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.751124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf6942-3bcb-4fc2-8df0-cb11f0be9b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.751395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf6942-3bcb-4fc2-8df0-cb11f0be9b44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.764557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf6942-3bcb-4fc2-8df0-cb11f0be9b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-dabf6942-3bcb-4fc2-8df0-cb11f0be9b44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:15.769137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:15.770119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:15.770936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 302/608] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:15.776909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf6942-3bcb-4fc2-8df0-cb11f0be9b44 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:15.795843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:15.845233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefc8b3-1d77-45b6-92e1-a2052fbbe513 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.875099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:15.887402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.888329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.888932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.889401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.890332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:15.890889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:15.892382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf6942-3bcb-4fc2-8df0-cb11f0be9b44 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:15.892755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:15.905512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 307/609] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:15.911014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:15.937439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:15.938768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:15.939242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:15.939726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:15.940714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:15.941232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:15.942882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:15.952492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 303/610] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:15.961871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:16.030449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87eb43-7dd2-49cc-942f-f9cbca6dee2c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:16.048807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:16.068835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2fbbe-b0cc-4df1-8691-e454fa5a6127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:16.069614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2fbbe-b0cc-4df1-8691-e454fa5a6127 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:16.069891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2fbbe-b0cc-4df1-8691-e454fa5a6127 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:16.070162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2fbbe-b0cc-4df1-8691-e454fa5a6127 None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/31183ac0fdd642e6a1a7578899aa9fdf/roles/72882169372f4db5bd4bf8c9c399b812` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:16.085989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2fbbe-b0cc-4df1-8691-e454fa5a6127 None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=31183ac0fdd642e6a1a7578899aa9fdf, role_id=72882169372f4db5bd4bf8c9c399b812)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:16.087433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2fbbe-b0cc-4df1-8691-e454fa5a6127 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:16.088545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0a596-402f-46c4-806b-a1d33c036251 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:16.101304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 308/611] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:15 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/31183ac0fdd642e6a1a7578899aa9fdf/roles/72882169372f4db5bd4bf8c9c399b812 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:16.136648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:16.137734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:16.138211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:16.138673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:16.155025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:16.156920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:16.171287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:16.188471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:16.189485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:16.190343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:16.190343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:16.190343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:16.193319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 304/612] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:16.201736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2fbbe-b0cc-4df1-8691-e454fa5a6127 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:16.365785 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2fbbe-b0cc-4df1-8691-e454fa5a6127 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:16.419096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:16.419711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:16.419983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:16.420252 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:16.420923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:16.421220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:16.422580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:16.441521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 309/613] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:16.447272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:16.579053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b611c9-0c24-43ac-80dd-7f3f1f22fc8d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:16.620812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f56643-629c-4b7e-b2e9-41310351383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:16.621374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f56643-629c-4b7e-b2e9-41310351383c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:16.621645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f56643-629c-4b7e-b2e9-41310351383c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:16.621912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f56643-629c-4b7e-b2e9-41310351383c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:16.630144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f56643-629c-4b7e-b2e9-41310351383c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:16.631568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f56643-629c-4b7e-b2e9-41310351383c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-66f56643-629c-4b7e-b2e9-41310351383c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:16.647501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:16.648478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:16.649421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 305/614] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:16.655069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:16.694052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:16.793446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934b4f5-fdd2-484e-b36e-6486425289e0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:16.843753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf4e26-0948-4590-aa67-143002e9ba0d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:16.844087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf4e26-0948-4590-aa67-143002e9ba0d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:16.844289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf4e26-0948-4590-aa67-143002e9ba0d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:16.845636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 306/615] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:01:16.850989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:16.851603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:16.851871 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:16.852141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:16.852888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:16.853188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:16.854721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:16.858930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d619d-264d-437c-8021-f875d864a51d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:16.859209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d619d-264d-437c-8021-f875d864a51d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:16.859478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d619d-264d-437c-8021-f875d864a51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:16.874435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 310/616] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Aug 9 19:01:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:16.879714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:16.909470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0f8d619d-264d-437c-8021-f875d864a51d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:16.910688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0f8d619d-264d-437c-8021-f875d864a51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:16.984054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:17.003519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209546d2-98ce-4fb9-9f67-00f6a7a0764b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:17.048505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:17.049413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.049963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.050246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.062450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:17.066172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:17.067650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:17.087110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:17.089266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:17.091458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:17.091458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:17.091458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:17.091458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 311/617] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:17.096068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:17.138226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 307/618] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:17.172346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d619d-264d-437c-8021-f875d864a51d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:17.219197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862c5a-2023-43a9-a9f6-2cf3548cba3d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:17.261368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:17.261982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.262258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.262522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.263162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:17.263455 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:17.266848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:17.273354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:17.279939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 312/619] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:17.285959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:17.425092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d619d-264d-437c-8021-f875d864a51d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:17.471021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdade9db-9ecf-4d58-b720-5e5ef3c245b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:17.471996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdade9db-9ecf-4d58-b720-5e5ef3c245b2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.472493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdade9db-9ecf-4d58-b720-5e5ef3c245b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.473841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdade9db-9ecf-4d58-b720-5e5ef3c245b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.490866 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdade9db-9ecf-4d58-b720-5e5ef3c245b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-bdade9db-9ecf-4d58-b720-5e5ef3c245b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:17.496888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:17.497967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:17.498807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:17.499453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63e4a5-6434-4d4f-ac66-0b62c9a35e7b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:17.501557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 308/620] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:17.502910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29212-6c99-4b40-8249-d113e70949c1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.503374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29212-6c99-4b40-8249-d113e70949c1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.503853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29212-6c99-4b40-8249-d113e70949c1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.505290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 309/621] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:17.509756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29212-6c99-4b40-8249-d113e70949c1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:17.541652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1fc0a-e0c0-4fde-98fc-f9767033d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:17.542650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1fc0a-e0c0-4fde-98fc-f9767033d1ef None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.543139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1fc0a-e0c0-4fde-98fc-f9767033d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.543588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1fc0a-e0c0-4fde-98fc-f9767033d1ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.550724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1fc0a-e0c0-4fde-98fc-f9767033d1ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:17.552397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1fc0a-e0c0-4fde-98fc-f9767033d1ef None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f2e1fc0a-e0c0-4fde-98fc-f9767033d1ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:17.565086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:17.566024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:17.566929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:17.566929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:17.566929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:17.566929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:17.566929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:17.566929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:17.566929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:17.566929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:17.568998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 313/622] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:17.570386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e6362-3ea6-4fa5-bd1f-35a8e910231e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.570867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e6362-3ea6-4fa5-bd1f-35a8e910231e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.571338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e6362-3ea6-4fa5-bd1f-35a8e910231e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.574173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:17.592835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1b0e6362-3ea6-4fa5-bd1f-35a8e910231e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:17.594423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1b0e6362-3ea6-4fa5-bd1f-35a8e910231e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:17.645006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29212-6c99-4b40-8249-d113e70949c1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:17.655733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:17.676585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 314/623] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:17.677883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e6362-3ea6-4fa5-bd1f-35a8e910231e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:17.698082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:17.699182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.699828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.700308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.702822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:17.703384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:17.706767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:17.718454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 310/624] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:17.719320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:17.826394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e6362-3ea6-4fa5-bd1f-35a8e910231e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:17.849844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97a82-1a1d-483e-91a4-fccb5e866206 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:17.864741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:17.867007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:17.867939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.868582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.869080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.870402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:17.871256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:17.873339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:17.886515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 315/625] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:17.887801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:17.895751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba230773-30c0-4e5f-a027-7e1b4b62804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:17.896720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230773-30c0-4e5f-a027-7e1b4b62804a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:17.897194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230773-30c0-4e5f-a027-7e1b4b62804a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:17.897692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230773-30c0-4e5f-a027-7e1b4b62804a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:17.914531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba230773-30c0-4e5f-a027-7e1b4b62804a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ba230773-30c0-4e5f-a027-7e1b4b62804a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:17.921540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:17.922486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:17.923308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:17.928042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 311/626] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:17.928885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba230773-30c0-4e5f-a027-7e1b4b62804a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:17.958184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:18.018598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2710-a5a4-4d2a-b17e-2350363a896d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.055058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba230773-30c0-4e5f-a027-7e1b4b62804a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.065722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.066379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.067093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.067379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.083571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:18.085790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:18.093847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631ea73-db23-4483-ae5c-64896285ac86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.095088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631ea73-db23-4483-ae5c-64896285ac86 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.095584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631ea73-db23-4483-ae5c-64896285ac86 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.096050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631ea73-db23-4483-ae5c-64896285ac86 None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/141d2c2f92c842feb6155aae1e9edfd2/roles/72882169372f4db5bd4bf8c9c399b812` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:18.104391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:18.105372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:18.106231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:18.106231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:18.106231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:18.108577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 316/627] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:18.109601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.117565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631ea73-db23-4483-ae5c-64896285ac86 None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=141d2c2f92c842feb6155aae1e9edfd2, role_id=72882169372f4db5bd4bf8c9c399b812)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:18.119966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631ea73-db23-4483-ae5c-64896285ac86 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:18.133800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 312/628] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:17 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/141d2c2f92c842feb6155aae1e9edfd2/roles/72882169372f4db5bd4bf8c9c399b812 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:18.135050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631ea73-db23-4483-ae5c-64896285ac86 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.169204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:18.237923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:18.241468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25afe-880b-4920-8c58-710c34d536ed None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.284881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bddd2-d922-4645-a563-592380d8489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.285491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bddd2-d922-4645-a563-592380d8489d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.285766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bddd2-d922-4645-a563-592380d8489d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.286033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bddd2-d922-4645-a563-592380d8489d None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.286755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bddd2-d922-4645-a563-592380d8489d None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:18.287056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bddd2-d922-4645-a563-592380d8489d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:18.289656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bddd2-d922-4645-a563-592380d8489d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:18.293723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631ea73-db23-4483-ae5c-64896285ac86 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.303024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 317/629] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:18.304839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2716a3-2786-45e0-aead-370f4b9fa741 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.305369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2716a3-2786-45e0-aead-370f4b9fa741 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.305893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2716a3-2786-45e0-aead-370f4b9fa741 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.308806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 318/630] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:18.311515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2716a3-2786-45e0-aead-370f4b9fa741 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.344007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d4ea0-51d1-4e1f-a5ee-70b5c04c7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.345121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d4ea0-51d1-4e1f-a5ee-70b5c04c7628 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.345689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d4ea0-51d1-4e1f-a5ee-70b5c04c7628 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.346203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d4ea0-51d1-4e1f-a5ee-70b5c04c7628 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.347032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d4ea0-51d1-4e1f-a5ee-70b5c04c7628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:18.347581 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d4ea0-51d1-4e1f-a5ee-70b5c04c7628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:18.349274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d4ea0-51d1-4e1f-a5ee-70b5c04c7628 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:18.363703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 313/631] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:18.365278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f583-2898-46d1-92c4-7910b4cdefcd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.365739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f583-2898-46d1-92c4-7910b4cdefcd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.366235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f583-2898-46d1-92c4-7910b4cdefcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.385993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9946f583-2898-46d1-92c4-7910b4cdefcd None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:18.387511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9946f583-2898-46d1-92c4-7910b4cdefcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:18.428950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2716a3-2786-45e0-aead-370f4b9fa741 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.470649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 314/632] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:18.471297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.471929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.472201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.472550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946f583-2898-46d1-92c4-7910b4cdefcd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.476492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.496383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:18.499228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:18.499969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:18.519768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:18.520792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:18.521680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:18.521680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:18.521680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:18.521680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 319/633] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:18.522340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.539976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:18.612031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946f583-2898-46d1-92c4-7910b4cdefcd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.649126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.649770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.650047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.650320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.656043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fc47d-12c7-4f48-b5be-9488cf3c2dd5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.657750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:18.659199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:18.672880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:18.673877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:18.674796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 315/634] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:18.675396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.705105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056de4c-7ef1-447c-98a8-47bd7b40dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.706199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056de4c-7ef1-447c-98a8-47bd7b40dca5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.707053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056de4c-7ef1-447c-98a8-47bd7b40dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.707472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056de4c-7ef1-447c-98a8-47bd7b40dca5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.721888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056de4c-7ef1-447c-98a8-47bd7b40dca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e056de4c-7ef1-447c-98a8-47bd7b40dca5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:18.730153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:18.731262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:18.732278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:18.739635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 320/635] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:18.739635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056de4c-7ef1-447c-98a8-47bd7b40dca5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.760691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:18.801002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a97f4d-6e36-4956-89fb-77152f3ac5b9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.824640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:18.839179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.839800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.840084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.840358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.841031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:18.841338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:18.842702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:18.854667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056de4c-7ef1-447c-98a8-47bd7b40dca5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:18.855611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 316/636] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:18.861549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.898909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fef7e4-b28b-4b4e-aa5f-291d52f963e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:18.899516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fef7e4-b28b-4b4e-aa5f-291d52f963e8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.899794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fef7e4-b28b-4b4e-aa5f-291d52f963e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.900063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fef7e4-b28b-4b4e-aa5f-291d52f963e8 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.901638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fef7e4-b28b-4b4e-aa5f-291d52f963e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:18.902083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fef7e4-b28b-4b4e-aa5f-291d52f963e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:18.903583 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fef7e4-b28b-4b4e-aa5f-291d52f963e8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:18.924185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 321/637] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:18.925435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27fe15-6065-40b3-adaa-338f1c0d1a24 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:18.925722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27fe15-6065-40b3-adaa-338f1c0d1a24 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:18.925993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27fe15-6065-40b3-adaa-338f1c0d1a24 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:18.927308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 322/638] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:18.927924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27fe15-6065-40b3-adaa-338f1c0d1a24 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:18.975894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517f1fe-29ce-4a30-9465-e18360251897 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.007269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:19.015181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.015852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.016120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.016399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.017137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:19.017440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:19.018818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:19.033624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 317/639] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:19.033624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.044148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27fe15-6065-40b3-adaa-338f1c0d1a24 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.081575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:19.087537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96900c36-f068-40fe-b9e2-a1599b8f20d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.088297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96900c36-f068-40fe-b9e2-a1599b8f20d2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.088605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96900c36-f068-40fe-b9e2-a1599b8f20d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.088991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96900c36-f068-40fe-b9e2-a1599b8f20d2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.098133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96900c36-f068-40fe-b9e2-a1599b8f20d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:19.099702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96900c36-f068-40fe-b9e2-a1599b8f20d2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-96900c36-f068-40fe-b9e2-a1599b8f20d2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:19.113117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:19.114122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:19.115087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 323/640] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:19.115729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0221114-9489-420a-9bf3-8cf2903c7a91 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.115729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0221114-9489-420a-9bf3-8cf2903c7a91 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.115868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0221114-9489-420a-9bf3-8cf2903c7a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.135107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a0221114-9489-420a-9bf3-8cf2903c7a91 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:19.137071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a0221114-9489-420a-9bf3-8cf2903c7a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:19.159669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec826573-578f-4fcd-896d-331984fb9c0e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.195653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f3ed0-0575-4974-ae08-883e80e943a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.196264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f3ed0-0575-4974-ae08-883e80e943a2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.196574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f3ed0-0575-4974-ae08-883e80e943a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.196848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f3ed0-0575-4974-ae08-883e80e943a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.217922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 324/641] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:19.218775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0221114-9489-420a-9bf3-8cf2903c7a91 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.295626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:19.310375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f3ed0-0575-4974-ae08-883e80e943a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:19.311838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f3ed0-0575-4974-ae08-883e80e943a2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:19.334268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0221114-9489-420a-9bf3-8cf2903c7a91 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.363464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:19.380837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.381207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.381588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.381966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.402364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:19.403846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:19.424298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:19.425408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:19.426313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:19.426313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:19.426313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:19.430178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 325/642] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:19.430851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.460690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 318/643] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:19.460690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f3ed0-0575-4974-ae08-883e80e943a2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.567588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:19.581488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f3ed0-0575-4974-ae08-883e80e943a2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.590268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208315c-8b4c-460d-92b8-2c6f9a2615fe None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.621951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b09c-0f6e-43cc-b80c-1a6a58a33ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.622585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b09c-0f6e-43cc-b80c-1a6a58a33ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.622862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b09c-0f6e-43cc-b80c-1a6a58a33ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.623132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b09c-0f6e-43cc-b80c-1a6a58a33ba5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.627647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.628659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.629127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.629694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.630710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:19.631206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:19.632877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:19.632987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220b09c-0f6e-43cc-b80c-1a6a58a33ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-0220b09c-0f6e-43cc-b80c-1a6a58a33ba5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:19.637503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:19.638437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 319/644] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:19.639328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b09c-0f6e-43cc-b80c-1a6a58a33ba5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.659650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 326/645] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:19.660750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.695159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:19.750869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b09c-0f6e-43cc-b80c-1a6a58a33ba5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.770417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc0155-3607-4906-bbbf-acecacf8a1b7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.785178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.786053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.786174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.786372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.787010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:19.787407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:19.788708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:19.805824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 320/646] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:19.805824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.813611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.814262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.814537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.814803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.820643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:19.826259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:19.827711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:19.843120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:19.844089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:19.845349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:19.845349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:19.845349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:19.845349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 327/647] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:19.845349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.919839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a723757-7e8f-4399-9fb6-1c7c9d82b76c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.933979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:19.955820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90cac-cb45-4fa5-b269-e5c49dafa206 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:19.966534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.967150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.967496 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.967767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:19.968461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:19.968765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:19.970126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:19.978530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 321/648] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:19.979391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:19.997963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089ae1a-7424-4c48-aef6-1b91fe729f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:19.998567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089ae1a-7424-4c48-aef6-1b91fe729f8d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:19.998842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089ae1a-7424-4c48-aef6-1b91fe729f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:19.999196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089ae1a-7424-4c48-aef6-1b91fe729f8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.068967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:20.073424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089ae1a-7424-4c48-aef6-1b91fe729f8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.075015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089ae1a-7424-4c48-aef6-1b91fe729f8d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.098360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2afcc-86a0-4bc6-ab01-90032cc01be7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.136207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.136951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.137232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.137497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.145641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.147122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:20.158706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:20.159700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 322/649] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:20.161077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.167114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:20.185546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 328/650] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:20.190955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089ae1a-7424-4c48-aef6-1b91fe729f8d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.272350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2dddb-0122-4c7a-8fc5-2542d0957d67 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.297552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089ae1a-7424-4c48-aef6-1b91fe729f8d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.308200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.308837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.309117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.309385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.310298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:20.310600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.311954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.313420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:20.324524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 323/651] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:20.325243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.337673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.338452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.338758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.339111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.349071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.350497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:20.365356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:20.371467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:20.372656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:20.372656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:20.372656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:20.372656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 329/652] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:20.372656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.409162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:20.425789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269f233-71e7-4c7e-8717-0c355cb7351a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.460885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73454697-735c-4947-bccc-39628f7f7930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.461490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454697-735c-4947-bccc-39628f7f7930 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.461775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454697-735c-4947-bccc-39628f7f7930 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.462049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454697-735c-4947-bccc-39628f7f7930 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.462782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73454697-735c-4947-bccc-39628f7f7930 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:20.463121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73454697-735c-4947-bccc-39628f7f7930 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.465032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73454697-735c-4947-bccc-39628f7f7930 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.474290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7d39d-d818-47ec-9663-1fd734bfa270 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.484011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 324/653] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:20.485376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9ffd8-7a37-4325-8b37-2a91ce00e480 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.485733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9ffd8-7a37-4325-8b37-2a91ce00e480 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.486076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9ffd8-7a37-4325-8b37-2a91ce00e480 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.487339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 325/654] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:20.488081 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9ffd8-7a37-4325-8b37-2a91ce00e480 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.512529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.513242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.513515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.513782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.524116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.525648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:20.539432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:20.540686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:20.542627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:20.542627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:20.542627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:20.542847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:20.542932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 330/655] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:20.542932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.593462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9ffd8-7a37-4325-8b37-2a91ce00e480 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.627723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.628339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.628644 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.628909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.635433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.636881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.639078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:20.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:20.648873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:20.649753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 326/656] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:20.650238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.656344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba31d4-7d71-4dc8-a1e4-8cecdaaebfd7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.692552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dcaca-6e95-4f11-83a2-3fafceb9030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.693168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dcaca-6e95-4f11-83a2-3fafceb9030a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.693446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dcaca-6e95-4f11-83a2-3fafceb9030a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.693718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dcaca-6e95-4f11-83a2-3fafceb9030a None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.694364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5dcaca-6e95-4f11-83a2-3fafceb9030a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:20.694665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5dcaca-6e95-4f11-83a2-3fafceb9030a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.696036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5dcaca-6e95-4f11-83a2-3fafceb9030a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.712554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 331/657] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:20.714716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f1510-788c-4f3a-a36f-addc2378b6c9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.715009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f1510-788c-4f3a-a36f-addc2378b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.715284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f1510-788c-4f3a-a36f-addc2378b6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.734178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-219f1510-788c-4f3a-a36f-addc2378b6c9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:20.735371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-219f1510-788c-4f3a-a36f-addc2378b6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:20.755686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99675e2-b359-426a-818d-a952e58599ae None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.768566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:20.790142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331dc2-a2c2-4536-93ca-aab4bd590f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.790740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331dc2-a2c2-4536-93ca-aab4bd590f79 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.791692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331dc2-a2c2-4536-93ca-aab4bd590f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.791985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331dc2-a2c2-4536-93ca-aab4bd590f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.802035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 332/658] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:20.802937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f1510-788c-4f3a-a36f-addc2378b6c9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.864535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9331dc2-a2c2-4536-93ca-aab4bd590f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.865902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9331dc2-a2c2-4536-93ca-aab4bd590f79 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.881982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:20.904124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f1510-788c-4f3a-a36f-addc2378b6c9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:20.938540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:20.939143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:20.939421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:20.939690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:20.940331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:20.940663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:20.942028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:20.953285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 333/659] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:20.954060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.967090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 327/660] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:20.967871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331dc2-a2c2-4536-93ca-aab4bd590f79 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:20.983805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.058926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7442a5-feb2-4a30-bebd-157ed3508f32 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.082666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331dc2-a2c2-4536-93ca-aab4bd590f79 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.100419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.101063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.101341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.101612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.102295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:21.102601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.104396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.111388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.117270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 334/661] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:21.118055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.124012 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.124754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.125032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.125308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.132774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.134207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:21.146298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:21.147420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 328/662] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:21.148587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.219770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.225967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f171b-c0e5-4ddd-8ab0-66cabf0f7049 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.249950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513bcd0-f0b1-4aa8-92a4-8df79bad3d14 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.259361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fe741-8140-40a9-9d59-ac62d9e0fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.260518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe741-8140-40a9-9d59-ac62d9e0fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.260883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe741-8140-40a9-9d59-ac62d9e0fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.261144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe741-8140-40a9-9d59-ac62d9e0fdb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.270864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fe741-8140-40a9-9d59-ac62d9e0fdb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-560fe741-8140-40a9-9d59-ac62d9e0fdb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:21.275449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:21.277424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 335/663] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:21.278336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fe741-8140-40a9-9d59-ac62d9e0fdb5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.287215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.287843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.288121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.288392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.296539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.298041 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:21.310368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:21.311421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:21.312294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 329/664] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:21.312802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.342244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.386190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fe741-8140-40a9-9d59-ac62d9e0fdb5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.423321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.423944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.424222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.424522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.425139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff7a01-f946-4c0c-95c8-5816d11485a7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.436348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.437826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:21.452602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:21.453721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:21.454630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:21.454630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:21.454630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:21.454630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 336/665] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:21.454630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.459108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.465560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.466164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.466435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.466704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.467424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=48410d57fe0d489cbb07cfca58ca7b30) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:21.467725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.469180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.474994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 330/666] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/role_assignments?user.id=48410d57fe0d489cbb07cfca58ca7b30&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:21.475749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.572458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf22ab2-19df-4c7c-9a8c-7b94d5eeafd8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.587803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.603401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd857c8-f5b6-411f-afe0-f950d6695a95 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.625740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.626663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.627121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.627575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.628722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:21.629234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.631720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.653660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.654270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.654553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.654839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.655598 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:21.655897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.657340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.660967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 337/667] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:21.662333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.666785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 331/668] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:21.667551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.737518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.789884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa54375-0e10-493e-b438-879fbb7fa746 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.799885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdcaf6-c01b-49eb-8b46-1a947ec7afc4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.828894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.829657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.830236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.830524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.831166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:21.831468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.832867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.838633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.839313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.839581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.839848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:21.840569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:21.840873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:21.842834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:21.846142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 332/669] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:21.851253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.860342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 338/670] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:21.860342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:21.893439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.956650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693f545-414b-4dd7-9bcc-603f5a643973 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.973169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa75ab-86f1-4af7-810b-4cec26198855 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:21.974794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:21.992952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e367f8-8c1e-4703-8847-ccc14683c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:21.993560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e367f8-8c1e-4703-8847-ccc14683c6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:21.993836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e367f8-8c1e-4703-8847-ccc14683c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:21.994144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e367f8-8c1e-4703-8847-ccc14683c6c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.009651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.010891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.011313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.011586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.022247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.023682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:22.043024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:22.044209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:22.045177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:22.045177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:22.045177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:22.045177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 339/671] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:22.045569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.081937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e367f8-8c1e-4703-8847-ccc14683c6c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.083506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e367f8-8c1e-4703-8847-ccc14683c6c9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.152575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:22.173709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa9ed8-4bb0-4dfb-a608-ae3dca8aedb5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.211590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.212482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.213547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.214057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.218480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 333/672] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:22.219389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e367f8-8c1e-4703-8847-ccc14683c6c9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.221790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.223391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:22.235594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:22.237070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:22.238111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:22.238111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:22.238111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:22.238111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:22.238111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:22.238111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:22.238111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:22.238111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:22.239734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 340/673] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:22.240670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.241493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:22.326899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e367f8-8c1e-4703-8847-ccc14683c6c9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.361909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.362676 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.363052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.363428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.364248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=92a88817f3a4402290e9358826a63904) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:22.364598 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.366015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.366264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcdd8-9b0b-4910-a7bf-4563c7c1dd95 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.380470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 334/674] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/role_assignments?user.id=92a88817f3a4402290e9358826a63904&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:22.380470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.419947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.420763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.421184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.421535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.422271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:22.422645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.424163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.436547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:22.441773 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 341/675] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:22.443795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.487490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:22.510860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4b2f8-4ec0-4844-8bc5-bb76d259978f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.555677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.557924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.558394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.559170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.573382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072d31c-f9e7-438a-91c3-a1d9bc84aea2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.575017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.577551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:22.593829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:22.594832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:22.595690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:22.595690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:22.595690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:22.595690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 335/676] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:22.596888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.627228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.627846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.628124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.628868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.629575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:22.629950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.631774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.652580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 342/677] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:22.653778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.726210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:22.739295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbdba9-357b-4218-94be-575eada73b3b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.772705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fb2ee-f60a-4650-920f-1dc47bcc0b22 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.787919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.788661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.788951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.789502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.801904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.804145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.810830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.811548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.811925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.812295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.812613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:22.813156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:22.813613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.815072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:22.818991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:22.820066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:22.821073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 336/678] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:22.821660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.835368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 343/679] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:22.836710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.931532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad582ad-f9b8-4cbf-8c02-e811677929ed None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.968946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:22.969266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67812e6-464b-4c6c-8bf4-3821985a8d91 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:22.969827 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:22.970224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:22.970596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:22.971422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=00be9b6d551a4ba39718a44101ecf9ca) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:22.971828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:22.973550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:22.980499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 337/680] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/role_assignments?user.id=00be9b6d551a4ba39718a44101ecf9ca&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:22.981526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:22.995451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:23.024275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:23.024910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.025209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.025562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.026291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:23.026590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:23.028015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:23.047642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 344/681] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:23.053386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:23.067943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:23.165103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb303-4219-4240-99e7-2d9b09ce96d6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:23.228010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:23.229421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.229707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.230249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.241078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:23.241638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf103a-797d-4e04-a4a0-1735b6e1975c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:23.242517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:23.258363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:23.261194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:23.262374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 338/682] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:23.268727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:23.284084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e73c8-25f8-4f8f-bae4-43c936e45924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:23.284784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e73c8-25f8-4f8f-bae4-43c936e45924 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.285059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e73c8-25f8-4f8f-bae4-43c936e45924 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.285369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e73c8-25f8-4f8f-bae4-43c936e45924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.353947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:23.376985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e73c8-25f8-4f8f-bae4-43c936e45924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:23.378449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e73c8-25f8-4f8f-bae4-43c936e45924 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:23.442268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fb131-8c4d-48e4-9948-e15334a24e54 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:23.470612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:23.486366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:23.487334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.487937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.488567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.489485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:23.490042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:23.491745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:23.505517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 345/683] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:23.510710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 339/684] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:23.513503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e73c8-25f8-4f8f-bae4-43c936e45924 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:23.521480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:23.606985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:23.631128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e73c8-25f8-4f8f-bae4-43c936e45924 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:23.674164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:23.674814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.675098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.675370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.702437 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:23.703901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:23.724713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:23.725708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:23.726869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:23.726869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:23.726869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:23.726869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 346/685] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:23.732563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:23.749543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168fbd4-110b-4b8b-aced-4cc56a1b3311 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:23.848083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9bc05-ac7f-4ae9-acfa-5f07b770ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:23.848634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9bc05-ac7f-4ae9-acfa-5f07b770ca47 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.848918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9bc05-ac7f-4ae9-acfa-5f07b770ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.849194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9bc05-ac7f-4ae9-acfa-5f07b770ca47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.849936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9bc05-ac7f-4ae9-acfa-5f07b770ca47 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=31183ac0fdd642e6a1a7578899aa9fdf) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:23.850244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9bc05-ac7f-4ae9-acfa-5f07b770ca47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:23.851677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9bc05-ac7f-4ae9-acfa-5f07b770ca47 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:23.864239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a3d5b-f678-4584-98cb-2e09c1d1cd16 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:23.874990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 340/686] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:23 2022] GET /identity/v3/role_assignments?user.id=31183ac0fdd642e6a1a7578899aa9fdf&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 608 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:23.880142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ac5a-8f43-4465-ae27-957390b893a1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.880426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ac5a-8f43-4465-ae27-957390b893a1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.880735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ac5a-8f43-4465-ae27-957390b893a1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.882378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 341/687] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:01:23.888058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113676ff-43ba-4d6d-a246-833c695aa785 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.888337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113676ff-43ba-4d6d-a246-833c695aa785 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.888660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113676ff-43ba-4d6d-a246-833c695aa785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.908247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-113676ff-43ba-4d6d-a246-833c695aa785 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:23.909540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-113676ff-43ba-4d6d-a246-833c695aa785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:23.917281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:23.927482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:23.928109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:23.928388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:23.928787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:23.929452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:23.929754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:23.934526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:23.958080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 347/688] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:23.965956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:23.978018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:24.035003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 342/689] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:01:24.063350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113676ff-43ba-4d6d-a246-833c695aa785 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:24.149335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c212e3-457e-46cd-bea8-a754f7d5956d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:24.188477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113676ff-43ba-4d6d-a246-833c695aa785 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:24.212631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:24.212631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.212631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.212631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.221264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:24.222732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:24.237384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:24.238311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:24.239187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 348/690] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:24.244932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:24.254653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c003abb-cea3-4e58-92bc-f2947dad0720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:24.256015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c003abb-cea3-4e58-92bc-f2947dad0720 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.256538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c003abb-cea3-4e58-92bc-f2947dad0720 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.257036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c003abb-cea3-4e58-92bc-f2947dad0720 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.269713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c003abb-cea3-4e58-92bc-f2947dad0720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-7c003abb-cea3-4e58-92bc-f2947dad0720 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:24.275040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:24.276107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:24.276997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:24.280017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 343/691] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:24.285994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c003abb-cea3-4e58-92bc-f2947dad0720 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:24.305093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:24.331191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:24.379717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613a9a3-a8a4-4e62-aac7-4d739263debc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:24.422359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c003abb-cea3-4e58-92bc-f2947dad0720 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:24.452985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:24.453601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.453930 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.454259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.455136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:24.455506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:24.457139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:24.466141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:24.466767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.467049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.467325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.468002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:24.468308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:24.469701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:24.488365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 344/692] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:24.495054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:24.633125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 349/693] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:24.638619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:24.722156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:24.734333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fef96-1418-448d-a858-ac3de50b41af None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:24.789059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:24.789642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.789923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.790193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.801499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:24.803117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:24.818480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:24.819447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:24.820800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:24.820800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:24.820800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:24.820800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 345/694] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:24.828462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:24.845174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6fcbe-ee3d-4b7e-add0-b5b9ccddf800 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:24.856734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:24.880847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb960e2-c3cb-403f-842c-6de9f0a28be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:24.881532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb960e2-c3cb-403f-842c-6de9f0a28be2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.881876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb960e2-c3cb-403f-842c-6de9f0a28be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.882213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb960e2-c3cb-403f-842c-6de9f0a28be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.882992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb960e2-c3cb-403f-842c-6de9f0a28be2 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=141d2c2f92c842feb6155aae1e9edfd2) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:24.883398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb960e2-c3cb-403f-842c-6de9f0a28be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:24.885484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb960e2-c3cb-403f-842c-6de9f0a28be2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:24.896266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 350/695] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:24 2022] GET /identity/v3/role_assignments?user.id=141d2c2f92c842feb6155aae1e9edfd2&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:24.897741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a7ae6-81e4-408b-8cdf-14552c7f6340 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.898086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a7ae6-81e4-408b-8cdf-14552c7f6340 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.898359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a7ae6-81e4-408b-8cdf-14552c7f6340 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.899888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 351/696] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:24.918922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335d18-920d-44df-a163-6297c27e83ea None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.919237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335d18-920d-44df-a163-6297c27e83ea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.919468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335d18-920d-44df-a163-6297c27e83ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.941408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d4a6f-a21c-4d96-8840-ea79f702fa48 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:24.951374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-82335d18-920d-44df-a163-6297c27e83ea None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:24.961991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-82335d18-920d-44df-a163-6297c27e83ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:24.985132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:24.985977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:24.986261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:24.986539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:24.987209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:24.987515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:24.988952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:25.015168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 346/697] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:25.022051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:25.037567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:25.105545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 352/698] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:25.136043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82335d18-920d-44df-a163-6297c27e83ea None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:25.140738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d759a-06c4-481f-94ac-ec26b4790d13 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:25.178886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:25.179663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:25.180028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:25.180305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:25.192863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:25.194452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:25.208315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:25.210413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:25.211702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 347/699] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:01:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:25.216356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:25.275814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82335d18-920d-44df-a163-6297c27e83ea None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:25.279266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:25.310351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:25.329814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:25.330445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:25.330849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:25.331142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:25.344664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:25.346319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:25.358660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403800c-0d2d-4998-9b77-8dd21599fd92 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:25.367718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:25.368759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:25.371217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:25.371217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:25.371217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:25.371217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 353/700] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:25.375372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:25.425267 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:25.425854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:25.426136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:25.426405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:25.427161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:25.427469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:25.429123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:25.466878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 348/701] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:01:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:25.472394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:25.515259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6db03-b114-4aff-9b90-7caf4dbae29f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:25.556220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:25.558714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:25.559504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:25.560430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:25.561333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:25.561842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:25.563629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:25.578137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 354/702] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:25.585781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:25.614459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:25.705575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c362d-ac6a-4a1c-8972-954eb83829c5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:25.709904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f75d4-dd9d-4864-84cc-49092bf17198 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:25.736927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:25.757826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df21f71-9f4c-4b97-8ec7-59a4a8f55178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:25.758665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b6144-5dd1-4312-8809-5d9d83563904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:25.758998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df21f71-9f4c-4b97-8ec7-59a4a8f55178 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:25.759285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b6144-5dd1-4312-8809-5d9d83563904 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:25.759568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b6144-5dd1-4312-8809-5d9d83563904 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:25.759846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df21f71-9f4c-4b97-8ec7-59a4a8f55178 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:25.759946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b6144-5dd1-4312-8809-5d9d83563904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:25.760375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df21f71-9f4c-4b97-8ec7-59a4a8f55178 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:25.761423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df21f71-9f4c-4b97-8ec7-59a4a8f55178 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:25.763059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df21f71-9f4c-4b97-8ec7-59a4a8f55178 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:25.790233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 355/703] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:01:25 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:01:25.792419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818413ca-9917-474a-affd-8af6c8317f06 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:25.792753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818413ca-9917-474a-affd-8af6c8317f06 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:25.793096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818413ca-9917-474a-affd-8af6c8317f06 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:25.796732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 356/704] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:25.800488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a840ac-a1a4-41bf-a869-f0e3e9fdaa6d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:25.801789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a840ac-a1a4-41bf-a869-f0e3e9fdaa6d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:25.803870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a840ac-a1a4-41bf-a869-f0e3e9fdaa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:25.831637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-19a840ac-a1a4-41bf-a869-f0e3e9fdaa6d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:25.833814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-19a840ac-a1a4-41bf-a869-f0e3e9fdaa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:25.853126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b6144-5dd1-4312-8809-5d9d83563904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:25.855548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b6144-5dd1-4312-8809-5d9d83563904 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:25.932162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 357/705] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:25.970509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a840ac-a1a4-41bf-a869-f0e3e9fdaa6d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:25.973639 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:26.011551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 349/706] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 19:01:26.017188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b6144-5dd1-4312-8809-5d9d83563904 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:26.069353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:26.166324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a840ac-a1a4-41bf-a869-f0e3e9fdaa6d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:26.174256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b6144-5dd1-4312-8809-5d9d83563904 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:26.214920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:26.215130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:26.215594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.215876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.216146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.216791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.217337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.217786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.218767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:26.220805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:26.228000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:26.230286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:26.235485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:26.236731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:26.237653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:26.241226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 358/707] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Aug 9 19:01:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:26.249426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:26.251120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:26.252226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:26.253311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:26.253311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:26.253311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:26.254988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 350/708] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:26.261446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:26.348944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:26.363350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:26.387606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd10ed-40ff-440f-917d-7117b5226eef None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:26.405501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbe94a-8a68-4540-b8a5-6b93191d9e6e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:26.425376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e3e95-feb2-4cab-9a3c-fb3a5ef0dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:26.425908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e3e95-feb2-4cab-9a3c-fb3a5ef0dcfc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.426189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e3e95-feb2-4cab-9a3c-fb3a5ef0dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.426495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e3e95-feb2-4cab-9a3c-fb3a5ef0dcfc None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.427195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e3e95-feb2-4cab-9a3c-fb3a5ef0dcfc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:26.427590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e3e95-feb2-4cab-9a3c-fb3a5ef0dcfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:26.429008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e3e95-feb2-4cab-9a3c-fb3a5ef0dcfc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:26.435244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 359/709] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Aug 9 19:01:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:26.436562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f5331-176e-42f1-9d6f-9ad0c08232a5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.436951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f5331-176e-42f1-9d6f-9ad0c08232a5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.437234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f5331-176e-42f1-9d6f-9ad0c08232a5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.438674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 360/710] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:26.443671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f5331-176e-42f1-9d6f-9ad0c08232a5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:26.456692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8d27f-2469-42cc-86cb-969ad4e8e5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:26.457311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8d27f-2469-42cc-86cb-969ad4e8e5bd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.457912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8d27f-2469-42cc-86cb-969ad4e8e5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.458190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8d27f-2469-42cc-86cb-969ad4e8e5bd None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.459125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8d27f-2469-42cc-86cb-969ad4e8e5bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:26.459429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8d27f-2469-42cc-86cb-969ad4e8e5bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:26.461371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8d27f-2469-42cc-86cb-969ad4e8e5bd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:26.478069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 351/711] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:26.479941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466bab-4126-4727-97ec-0e0874b9d960 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.480232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466bab-4126-4727-97ec-0e0874b9d960 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.480623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466bab-4126-4727-97ec-0e0874b9d960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.502928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-07466bab-4126-4727-97ec-0e0874b9d960 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:26.504464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-07466bab-4126-4727-97ec-0e0874b9d960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:26.558866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f5331-176e-42f1-9d6f-9ad0c08232a5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:26.591290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:26.599098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:26.600055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.600525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.600952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.601739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:26.602010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 352/712] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:26.602460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:26.602946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07466bab-4126-4727-97ec-0e0874b9d960 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:26.605002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:26.612243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 361/713] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Aug 9 19:01:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:26.629266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:26.729690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:26.802348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07466bab-4126-4727-97ec-0e0874b9d960 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:26.817691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a9784-69d8-48bb-81c6-6815f3f98233 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:26.846571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:26.847244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.847523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.847796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.855419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:26.856901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:26.867443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:26.867443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:26.867443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:26.867443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] PATH_INFO: `/v3/services/network` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:26.867818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:26.868215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:26.874927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:26.875905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:26.876887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:26.876887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:26.876887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:26.876887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:26.876887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:26.876887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:26.876887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:26.876887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:26.884108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:26.886080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:26.886911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:26.887656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 353/714] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:26.887656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:26.889719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 362/715] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:26 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:26.895702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:26.998209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:27.045628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64cdae-3922-4a9f-be6e-b304a5daa2cb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:27.074056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:27.083584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:27.084542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:27.084829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:27.085098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:27.085788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:27.086090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:27.087518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:27.101940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 354/716] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:27.107732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:27.112137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf47b8-f670-4d7e-a50e-8ab07eff1f73 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:27.176514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:27.177380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:27.177657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:27.177925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:27.179440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] RBAC: Adding query filter params (name=network) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:27.179765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:27.181172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:27.191591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 363/717] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:27.192891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:27.303622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4e7c-a767-4718-ab93-51dd0091dfb9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:27.356348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a1df1-28d7-4557-ada0-05e3c761901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:27.357000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a1df1-28d7-4557-ada0-05e3c761901c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:27.357277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a1df1-28d7-4557-ada0-05e3c761901c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:27.357547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a1df1-28d7-4557-ada0-05e3c761901c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:27.358273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a1df1-28d7-4557-ada0-05e3c761901c None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=48410d57fe0d489cbb07cfca58ca7b30) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:27.358576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a1df1-28d7-4557-ada0-05e3c761901c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:27.360007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a1df1-28d7-4557-ada0-05e3c761901c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:27.365890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 355/718] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:27 2022] GET /identity/v3/role_assignments?user.id=48410d57fe0d489cbb07cfca58ca7b30&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:27.388508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c20067-05d9-4a2c-9acf-14b0c59872ac None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:27.415413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:27.432148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d1f80-6fc8-4950-bcc9-324b44cd09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:27.432909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d1f80-6fc8-4950-bcc9-324b44cd09d1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:27.433186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d1f80-6fc8-4950-bcc9-324b44cd09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:27.433457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d1f80-6fc8-4950-bcc9-324b44cd09d1 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:27.434105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d1f80-6fc8-4950-bcc9-324b44cd09d1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:27.434405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d1f80-6fc8-4950-bcc9-324b44cd09d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:27.436082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d1f80-6fc8-4950-bcc9-324b44cd09d1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:27.449520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 364/719] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:27.596457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ebab-3221-484b-80de-cae9bd7041cd None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:27.597066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ebab-3221-484b-80de-cae9bd7041cd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:27.597551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783ebab-3221-484b-80de-cae9bd7041cd None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:27.599037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 356/720] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:27.605144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb1a6-b334-4d78-a88f-4325a4ec6596 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:27.605774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb1a6-b334-4d78-a88f-4325a4ec6596 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:27.606247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb1a6-b334-4d78-a88f-4325a4ec6596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:27.626969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fa8eb1a6-b334-4d78-a88f-4325a4ec6596 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:27.628573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fa8eb1a6-b334-4d78-a88f-4325a4ec6596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:27.703634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 365/721] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:01:27.726332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783ebab-3221-484b-80de-cae9bd7041cd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:27.781926 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:27.835655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783ebab-3221-484b-80de-cae9bd7041cd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:27.887500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75c446-cc15-4a44-8902-2ac2784a45a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:27.888051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c446-cc15-4a44-8902-2ac2784a45a3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:27.888327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c446-cc15-4a44-8902-2ac2784a45a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:27.888637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c446-cc15-4a44-8902-2ac2784a45a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:27.899233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75c446-cc15-4a44-8902-2ac2784a45a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-bb75c446-cc15-4a44-8902-2ac2784a45a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:27.905239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:27.906172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:27.906983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 357/722] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:27.912723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8eb1a6-b334-4d78-a88f-4325a4ec6596 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:28.049543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8eb1a6-b334-4d78-a88f-4325a4ec6596 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:28.074254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:28.096749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:28.097358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:28.097630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:28.097894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:28.098560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:28.098855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:28.100207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:28.111847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 366/723] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:28.117268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75c446-cc15-4a44-8902-2ac2784a45a3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:28.251266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75c446-cc15-4a44-8902-2ac2784a45a3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:28.265800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:28.303503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:28.304139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:28.304411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:28.304703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:28.320710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:28.322190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:28.341074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:28.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:28.343017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:28.343017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:28.343017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:28.343017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 358/724] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:28.348060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:28.476454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aaffd-ea91-4f7f-94fc-336335b7609b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:28.522223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:28.522744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:28.523024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:28.523295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:28.524014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:28.524311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:28.525820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:28.544384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 367/725] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:28.549466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:28.717777 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320a834-220d-4a55-9ed6-60b588ee8b8a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:28.761867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c65471-9c0e-4237-b517-106dc8ac7774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:28.762497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c65471-9c0e-4237-b517-106dc8ac7774 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:28.762772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c65471-9c0e-4237-b517-106dc8ac7774 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:28.763038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c65471-9c0e-4237-b517-106dc8ac7774 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:28.773373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c65471-9c0e-4237-b517-106dc8ac7774 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:28.774746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c65471-9c0e-4237-b517-106dc8ac7774 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-93c65471-9c0e-4237-b517-106dc8ac7774 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:28.785945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:28.786906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:28.787825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 359/726] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:28.793608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:28.823022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010424a-eda7-4aab-a932-de61005b69e4 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:28.823334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010424a-eda7-4aab-a932-de61005b69e4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:28.823568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010424a-eda7-4aab-a932-de61005b69e4 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:28.825049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 360/727] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:01:28.837797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5dae0-1040-4256-8b5d-8d77a4067080 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:28.838124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5dae0-1040-4256-8b5d-8d77a4067080 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:28.838398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5dae0-1040-4256-8b5d-8d77a4067080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:28.864094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:28.868802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-89b5dae0-1040-4256-8b5d-8d77a4067080 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:28.873294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-89b5dae0-1040-4256-8b5d-8d77a4067080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:28.904612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:28.952492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa43898-df49-4ece-8772-0cb73eebfe07 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:28.994163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda1737-e463-422e-9439-35e76c00bf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:28.995328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda1737-e463-422e-9439-35e76c00bf54 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:28.996700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda1737-e463-422e-9439-35e76c00bf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:28.999719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda1737-e463-422e-9439-35e76c00bf54 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.000711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda1737-e463-422e-9439-35e76c00bf54 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:29.001233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda1737-e463-422e-9439-35e76c00bf54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:29.002868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda1737-e463-422e-9439-35e76c00bf54 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:29.035160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 368/728] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Aug 9 19:01:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:29.035441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 361/729] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 19:01:29.045506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5dae0-1040-4256-8b5d-8d77a4067080 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:29.050810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca754bff-1050-465e-a641-c39f932b3ccd None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.051330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca754bff-1050-465e-a641-c39f932b3ccd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.051806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca754bff-1050-465e-a641-c39f932b3ccd None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.053613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 369/730] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:29.060564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83267a-9b29-4a00-b59e-900f54992129 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.061102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83267a-9b29-4a00-b59e-900f54992129 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.061766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83267a-9b29-4a00-b59e-900f54992129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.083031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4c83267a-9b29-4a00-b59e-900f54992129 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:29.084712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4c83267a-9b29-4a00-b59e-900f54992129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:29.160735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 370/731] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:29.161904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83267a-9b29-4a00-b59e-900f54992129 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:29.194145 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:29.195477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5dae0-1040-4256-8b5d-8d77a4067080 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:29.253534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cba9ad-231e-4c30-84ed-5904fddf14a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:29.254847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cba9ad-231e-4c30-84ed-5904fddf14a6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.255643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cba9ad-231e-4c30-84ed-5904fddf14a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.256317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cba9ad-231e-4c30-84ed-5904fddf14a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.279414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83267a-9b29-4a00-b59e-900f54992129 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:29.286602 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:29.324288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfc634-01e1-43f1-84c4-e080dcfe3ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:29.327555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfc634-01e1-43f1-84c4-e080dcfe3ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.327980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfc634-01e1-43f1-84c4-e080dcfe3ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.328411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfc634-01e1-43f1-84c4-e080dcfe3ed8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.339604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfc634-01e1-43f1-84c4-e080dcfe3ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d3cfc634-01e1-43f1-84c4-e080dcfe3ed8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:29.344967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:29.345912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:29.346775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:29.348233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cba9ad-231e-4c30-84ed-5904fddf14a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:29.354579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cba9ad-231e-4c30-84ed-5904fddf14a6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:29.355209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 371/732] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:29.356248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfc634-01e1-43f1-84c4-e080dcfe3ed8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:29.453436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:29.471241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfc634-01e1-43f1-84c4-e080dcfe3ed8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:29.494601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 362/733] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 19:01:29.494601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cba9ad-231e-4c30-84ed-5904fddf14a6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:29.515234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51a8f0-bd60-4048-bd69-15d55c630228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:29.516608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51a8f0-bd60-4048-bd69-15d55c630228 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.517090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51a8f0-bd60-4048-bd69-15d55c630228 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.517557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51a8f0-bd60-4048-bd69-15d55c630228 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.518512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51a8f0-bd60-4048-bd69-15d55c630228 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:29.520139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51a8f0-bd60-4048-bd69-15d55c630228 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:29.546581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 372/734] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:01:29 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:29.548000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51a8f0-bd60-4048-bd69-15d55c630228 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:29.554480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:29.609544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cba9ad-231e-4c30-84ed-5904fddf14a6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:29.668127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:29.668127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.668127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.668127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.668621 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:29.668621 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:29.668621 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:29.682639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 363/735] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:29.689577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:29.719609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51a8f0-bd60-4048-bd69-15d55c630228 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:29.757982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:29.759056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.759719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.760274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.772689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:29.774631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:29.785063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:29.795126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:29.796133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:29.797154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:29.797154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:29.797154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:29.797154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 373/736] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:29.802501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:29.895564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202a2a6-416f-4485-a0e1-e21692ae275e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:29.908230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:29.943705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff34c6c-013c-442c-86c7-28907806d3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:29.944636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff34c6c-013c-442c-86c7-28907806d3bd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.945054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff34c6c-013c-442c-86c7-28907806d3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.945466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff34c6c-013c-442c-86c7-28907806d3bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.946544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04b333-3102-4fb1-a67b-9c0ccd2325fe None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:29.957286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff34c6c-013c-442c-86c7-28907806d3bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:29.958737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff34c6c-013c-442c-86c7-28907806d3bd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-9ff34c6c-013c-442c-86c7-28907806d3bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:29.976634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:29.977594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:29.979141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:29.979141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:29.979141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:29.979141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 364/737] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:29.979988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709370-70ce-4395-82c3-38c33a783c48 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.980268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709370-70ce-4395-82c3-38c33a783c48 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.984715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709370-70ce-4395-82c3-38c33a783c48 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.986198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 365/738] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:29.987105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af886414-059c-40c5-aae5-7793f52a8b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:29.987517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4709370-70ce-4395-82c3-38c33a783c48 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:29.987960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af886414-059c-40c5-aae5-7793f52a8b2c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:29.988314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af886414-059c-40c5-aae5-7793f52a8b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:29.988678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af886414-059c-40c5-aae5-7793f52a8b2c None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:29.989395 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af886414-059c-40c5-aae5-7793f52a8b2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:29.989766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af886414-059c-40c5-aae5-7793f52a8b2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:29.992332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af886414-059c-40c5-aae5-7793f52a8b2c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.012534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 374/739] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:30.013916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6f5e-ffca-4dd5-94e6-8b4dcceaeb7b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.014195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6f5e-ffca-4dd5-94e6-8b4dcceaeb7b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.014468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6f5e-ffca-4dd5-94e6-8b4dcceaeb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.033125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-eced6f5e-ffca-4dd5-94e6-8b4dcceaeb7b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:30.034510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eced6f5e-ffca-4dd5-94e6-8b4dcceaeb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:30.093165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:30.106349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4709370-70ce-4395-82c3-38c33a783c48 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.116458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 375/740] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:30.116458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced6f5e-ffca-4dd5-94e6-8b4dcceaeb7b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.153665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f74b0f-5d13-48fd-81c7-b93b56ab8004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.154276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74b0f-5d13-48fd-81c7-b93b56ab8004 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.154559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74b0f-5d13-48fd-81c7-b93b56ab8004 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.154833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74b0f-5d13-48fd-81c7-b93b56ab8004 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.155554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f74b0f-5d13-48fd-81c7-b93b56ab8004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:30.155863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f74b0f-5d13-48fd-81c7-b93b56ab8004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:30.157282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f74b0f-5d13-48fd-81c7-b93b56ab8004 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.170700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 366/741] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:30.171721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43822e6-7dc2-4c99-9227-0ca6db778a5e None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.172006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43822e6-7dc2-4c99-9227-0ca6db778a5e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.172278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43822e6-7dc2-4c99-9227-0ca6db778a5e None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.177029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 367/742] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:30.177029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43822e6-7dc2-4c99-9227-0ca6db778a5e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.204349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:30.225159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced6f5e-ffca-4dd5-94e6-8b4dcceaeb7b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.260666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.261296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.261569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.261832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.268383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:30.269903 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:30.281132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:30.282022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 376/743] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:30.282916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.297378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43822e6-7dc2-4c99-9227-0ca6db778a5e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.336028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:30.344916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f5fd5-964a-40dc-8d1f-b191660a2dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.345911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5fd5-964a-40dc-8d1f-b191660a2dfa None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.346630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5fd5-964a-40dc-8d1f-b191660a2dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.347334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5fd5-964a-40dc-8d1f-b191660a2dfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.361394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f5fd5-964a-40dc-8d1f-b191660a2dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d86f5fd5-964a-40dc-8d1f-b191660a2dfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:30.367874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:30.369708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:30.370695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:30.373327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 368/744] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:30.375083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd30f8-4cc3-4ea9-b551-cc0f66956729 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.375526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd30f8-4cc3-4ea9-b551-cc0f66956729 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.375976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd30f8-4cc3-4ea9-b551-cc0f66956729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.390043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805abcd-d670-4840-96b5-78b2f3b03103 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.401399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5dfd30f8-4cc3-4ea9-b551-cc0f66956729 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:30.404261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5dfd30f8-4cc3-4ea9-b551-cc0f66956729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:30.427120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.427743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.428020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.428293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.436231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:30.437715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:30.448835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:30.449846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 377/745] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:30.450801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.492785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 369/746] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 19:01:30.496461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd30f8-4cc3-4ea9-b551-cc0f66956729 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.513740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:30.564827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cebeb-b962-4311-a169-d4a609d8a294 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.575126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:30.601830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.602444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.602719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.602989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.603939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:30.604243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:30.605705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.606936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd30f8-4cc3-4ea9-b551-cc0f66956729 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.624159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 378/747] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:30.624159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.644060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.644988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.645446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.645896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.646734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:30.647216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:30.648812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.659496 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 370/748] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:30.660653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.731157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e20e68-af4b-4b79-89a9-348c5b02d370 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.743317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:30.768837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.769505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.769788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.770060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.770817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:30.771123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:30.772270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10683143-d3d5-4495-8bba-0954e6cfc49f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.772569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.790303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 379/749] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:30.791519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.817361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70360a75-10fe-4674-a1be-952f7d50ff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.817980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70360a75-10fe-4674-a1be-952f7d50ff74 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.818261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70360a75-10fe-4674-a1be-952f7d50ff74 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.818537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70360a75-10fe-4674-a1be-952f7d50ff74 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.819178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70360a75-10fe-4674-a1be-952f7d50ff74 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:30.820321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:30.820576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70360a75-10fe-4674-a1be-952f7d50ff74 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.839163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 371/750] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:01:30 2022] POST /identity/v3/services => generated 240 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:30.840673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70360a75-10fe-4674-a1be-952f7d50ff74 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:30.917085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d163282-b5dc-49ab-bae3-5eb9a55d6e60 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:30.977836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:30.979078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:30.979415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:30.979685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:30.980685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=141d2c2f92c842feb6155aae1e9edfd2) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:30.981045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:30.982626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:30.991376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 380/751] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/role_assignments?user.id=141d2c2f92c842feb6155aae1e9edfd2&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:30.992309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:31.109281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:31.128211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23ebd2-f6d6-48ba-a424-ef2429e6a7df None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:31.147300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70360a75-10fe-4674-a1be-952f7d50ff74 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:31.168268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884cd7a-ef85-4518-884f-fe632d8d59ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:31.168846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884cd7a-ef85-4518-884f-fe632d8d59ef None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:31.169129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884cd7a-ef85-4518-884f-fe632d8d59ef None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:31.169403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884cd7a-ef85-4518-884f-fe632d8d59ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:31.195536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:31.196751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:31.197249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:31.197698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:31.201323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:31.222016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:31.225358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:31.249051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:31.250063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:31.250913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:31.250913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:31.250913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:31.253193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 372/752] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:31.256244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884cd7a-ef85-4518-884f-fe632d8d59ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:31.260576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884cd7a-ef85-4518-884f-fe632d8d59ef None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:31.260742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:31.366276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:31.381397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 381/753] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:01:31.391426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884cd7a-ef85-4518-884f-fe632d8d59ef None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:31.528813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b28839-415e-4d59-a9aa-18e1b641013f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:31.606217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:31.607180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884cd7a-ef85-4518-884f-fe632d8d59ef None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:31.608324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:31.610665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:31.611116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:31.612016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:31.612750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:31.614995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:31.647253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 373/754] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:31.661735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:31.707618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:31.708557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:31.719040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:31.719421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:31.749481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:31.750152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:31.750945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:31.794753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:31.795695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:31.796594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:31.796594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:31.796594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:31.796594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 382/755] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:31.805458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:31.839203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7b4bc-ca7e-4c8d-828a-e83394305d1a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:31.888473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:31.889128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:31.889403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:31.889670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:31.899252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:31.902082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:31.918072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:31.919072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:31.920002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 374/756] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:01:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:31.927592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:31.964818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:32.095159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec31cdb-cbfa-4c9a-90e0-7b5fa885e0c3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.138605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219efd7e-5f1b-4287-bfaa-baa5756739d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.139188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efd7e-5f1b-4287-bfaa-baa5756739d7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.139468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efd7e-5f1b-4287-bfaa-baa5756739d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.139741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219efd7e-5f1b-4287-bfaa-baa5756739d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.140594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219efd7e-5f1b-4287-bfaa-baa5756739d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:32.140900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219efd7e-5f1b-4287-bfaa-baa5756739d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.142578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219efd7e-5f1b-4287-bfaa-baa5756739d7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:32.158284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f2de1-3dbc-4ea3-8015-2d993f899d98 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.158465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 383/757] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:32.159624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7bf8f-0fd5-4b93-9463-18e9d33bd092 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.160078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7bf8f-0fd5-4b93-9463-18e9d33bd092 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.160356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7bf8f-0fd5-4b93-9463-18e9d33bd092 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.162672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 384/758] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:32.164338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7bf8f-0fd5-4b93-9463-18e9d33bd092 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:32.171425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:32.212417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40eb81-0c68-4a9c-b2f1-726148d39ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.214164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40eb81-0c68-4a9c-b2f1-726148d39ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.214539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40eb81-0c68-4a9c-b2f1-726148d39ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.214984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40eb81-0c68-4a9c-b2f1-726148d39ac4 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.215842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40eb81-0c68-4a9c-b2f1-726148d39ac4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:32.216270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40eb81-0c68-4a9c-b2f1-726148d39ac4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.217951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40eb81-0c68-4a9c-b2f1-726148d39ac4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:32.238387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 375/759] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:01:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:32.239894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931d05e-327a-4c22-851a-5c83acf164ff None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.240242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931d05e-327a-4c22-851a-5c83acf164ff None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.240609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931d05e-327a-4c22-851a-5c83acf164ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.260096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6931d05e-327a-4c22-851a-5c83acf164ff None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:32.261862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6931d05e-327a-4c22-851a-5c83acf164ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:32.282516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:32.288931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7bf8f-0fd5-4b93-9463-18e9d33bd092 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.336855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.337934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.338750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.339156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.348061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.349523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:32.352699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 376/760] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:32.354337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931d05e-327a-4c22-851a-5c83acf164ff None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:32.362440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:32.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:32.365537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 385/761] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:32.377434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:32.480888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:32.488609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931d05e-327a-4c22-851a-5c83acf164ff None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.490079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0dbee4-3683-488c-91df-164aac72b410 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.525554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.526327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.526680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.527023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.527781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:32.528157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.528582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.529216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.529492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.529761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.529923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:32.540630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.542637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:32.543662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 386/762] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:32.544607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:32.564372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:32.565398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:32.566285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:32.566285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:32.566285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:32.570309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 377/763] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:32.571153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:32.572324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:32.659336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb6aa2-2834-4743-b331-f1bf10902364 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.680318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de09540-0651-4bff-98b7-2b14bdc2e41f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.697991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.698805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.699255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.699631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.700316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.701930 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:32.711577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:32.713109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:32.714054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 387/764] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Aug 9 19:01:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:32.714815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:32.717017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.717663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.718013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.718284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.719113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=87b647f3da7246b0b530a65c500d5064) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:32.719695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.721240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:32.727837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 378/765] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:32 2022] GET /identity/v3/role_assignments?user.id=87b647f3da7246b0b530a65c500d5064&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:32.737270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:32.757937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:32.818693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:32.831524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25becac9-e131-4539-9aaa-b68fe235a23a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.877588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.878331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.878648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.878920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.879589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:32.879890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.881328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:32.915462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 388/766] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:32.918598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:32.950511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86ac48-cad9-4b67-a2d1-06b406927222 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:32.991325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:32.991941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:32.992222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:32.992526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:32.993160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:32.993463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:32.995036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:33.000380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 379/767] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Aug 9 19:01:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:33.005608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:33.162748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:33.194509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c7391-a593-4853-9c24-fed36646b3fb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:33.195080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6484a0-c99c-445f-9120-41ed3c3022fa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:33.233629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:33.234588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.234872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.235140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.236062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:33.236362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:33.237919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:33.245494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 380/768] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Aug 9 19:01:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:33.251895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:33.292024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:33.293123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.294230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.294708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.317401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:33.318882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:33.338568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:33.339492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:33.340383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:33.340383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:33.340383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:33.340383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:33.340383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:33.340383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:33.340383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:33.340383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:33.344589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 389/769] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:33.348722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:33.353840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:33.366603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca03343-c386-43d0-9057-cb2735f2df6e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:33.411858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de83dc-11f4-4910-8fac-2598f18f12e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:33.412908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de83dc-11f4-4910-8fac-2598f18f12e6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.413190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de83dc-11f4-4910-8fac-2598f18f12e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.413732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de83dc-11f4-4910-8fac-2598f18f12e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.414442 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de83dc-11f4-4910-8fac-2598f18f12e6 None admin] RBAC: Adding query filter params (interface=public, service_id=66ffc3486906409d8f5722915039d802, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:33.414996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de83dc-11f4-4910-8fac-2598f18f12e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:33.417874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de83dc-11f4-4910-8fac-2598f18f12e6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:33.428848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 381/770] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Aug 9 19:01:33 2022] GET /identity/v3/endpoints?service_id=66ffc3486906409d8f5722915039d802&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:33.430104 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b25d5-e672-444f-902e-c853ad6b7155 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.430381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b25d5-e672-444f-902e-c853ad6b7155 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.430658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b25d5-e672-444f-902e-c853ad6b7155 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.438034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 382/771] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:33.438906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b25d5-e672-444f-902e-c853ad6b7155 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:33.444126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:33.471741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d476-94bc-4df8-b313-07cc2591be2e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:33.511586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c47e8d-6221-470b-b207-9dd2cf465ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:33.512464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c47e8d-6221-470b-b207-9dd2cf465ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.512887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c47e8d-6221-470b-b207-9dd2cf465ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.513299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c47e8d-6221-470b-b207-9dd2cf465ba6 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.514196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c47e8d-6221-470b-b207-9dd2cf465ba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:33.514700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c47e8d-6221-470b-b207-9dd2cf465ba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:33.516269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c47e8d-6221-470b-b207-9dd2cf465ba6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:33.528788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 390/772] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:33.530199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6d3f-3988-4282-990a-80d66959ac79 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.530476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6d3f-3988-4282-990a-80d66959ac79 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.530741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6d3f-3988-4282-990a-80d66959ac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.548106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-556e6d3f-3988-4282-990a-80d66959ac79 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:33.549288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-556e6d3f-3988-4282-990a-80d66959ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:33.567735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b25d5-e672-444f-902e-c853ad6b7155 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:33.606143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3163-0121-4074-9126-169b7583b3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:33.606754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3163-0121-4074-9126-169b7583b3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.607035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3163-0121-4074-9126-169b7583b3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.607303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3163-0121-4074-9126-169b7583b3b7 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.607910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac3163-0121-4074-9126-169b7583b3b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:33.609321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac3163-0121-4074-9126-169b7583b3b7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:33.614881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:33.624984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 383/773] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:01:33 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:33.626201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac051fb2-674b-45c5-8a5f-754c393bd3fd None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.626596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac051fb2-674b-45c5-8a5f-754c393bd3fd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.626878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac051fb2-674b-45c5-8a5f-754c393bd3fd None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.635938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 391/774] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:33.636127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 384/775] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:33.636204 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e6d3f-3988-4282-990a-80d66959ac79 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:33.636274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41295017-6a8b-4463-b0ad-c9ba1e59e5d3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.636274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41295017-6a8b-4463-b0ad-c9ba1e59e5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.636274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41295017-6a8b-4463-b0ad-c9ba1e59e5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.675770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-41295017-6a8b-4463-b0ad-c9ba1e59e5d3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:33.677020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-41295017-6a8b-4463-b0ad-c9ba1e59e5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:33.753167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e6d3f-3988-4282-990a-80d66959ac79 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:33.776715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:33.788348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 385/776] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:33.796461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41295017-6a8b-4463-b0ad-c9ba1e59e5d3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:33.814488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d067-7907-4e8d-afd6-a50c27ff4b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:33.815155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d067-7907-4e8d-afd6-a50c27ff4b81 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.815427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d067-7907-4e8d-afd6-a50c27ff4b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.815695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d067-7907-4e8d-afd6-a50c27ff4b81 None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/48410d57fe0d489cbb07cfca58ca7b30/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.817627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1d067-7907-4e8d-afd6-a50c27ff4b81 None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=48410d57fe0d489cbb07cfca58ca7b30, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:33.819045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1d067-7907-4e8d-afd6-a50c27ff4b81 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:33.834614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 392/777] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:33 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/48410d57fe0d489cbb07cfca58ca7b30/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:33.835443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d067-7907-4e8d-afd6-a50c27ff4b81 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:33.895968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:33.922878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41295017-6a8b-4463-b0ad-c9ba1e59e5d3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:33.960593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:33.961242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:33.961590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:33.961841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:33.962441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:33.964552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:33.965135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d067-7907-4e8d-afd6-a50c27ff4b81 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:33.978330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:33.979530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:33.980376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:33.986358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 386/778] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:33 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:33.987434 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:34.016494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236844-75ed-4e7e-827e-6b9593c1de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:34.018207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236844-75ed-4e7e-827e-6b9593c1de0e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:34.019030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236844-75ed-4e7e-827e-6b9593c1de0e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:34.019524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87236844-75ed-4e7e-827e-6b9593c1de0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:34.037462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87236844-75ed-4e7e-827e-6b9593c1de0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-87236844-75ed-4e7e-827e-6b9593c1de0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:34.044161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:34.045171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:34.046060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:34.049096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 393/779] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:34.054478 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236844-75ed-4e7e-827e-6b9593c1de0e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:34.073024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:34.140723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cfa4d-3051-49eb-b2b4-943a4637fac8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:34.192844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:34.193467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:34.193749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:34.194026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:34.194668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] RBAC: Adding query filter params (name=network) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:34.194974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:34.195260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87236844-75ed-4e7e-827e-6b9593c1de0e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:34.196341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:34.201945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 387/780] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:34.210641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:34.231448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:34.259565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:34.260233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:34.260545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:34.260819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:34.261494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:34.261952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:34.263481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:34.273827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 394/781] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:34.279931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:34.365939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7859f7-4fdf-43d7-8958-65f6af513a94 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:34.406042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:34.419695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:34.420552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:34.421404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:34.421853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:34.422937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:34.423388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:34.425444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:34.433236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 388/782] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:34.439135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:34.485281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f826ed-2ab5-4666-a338-7419d426212c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:34.541945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:34.542935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:34.543406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:34.543890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:34.560505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:34.562834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:34.594521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:34.595515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:34.596373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:34.596373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:34.596373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:34.598513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 395/783] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:34.604306 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:34.646195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8264a-3629-446b-8872-d6a3419e30c1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:34.667015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:34.757759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:34.758765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:34.759248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:34.759751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:34.760683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] RBAC: Adding query filter params (interface=public, service_id=cbe42941a671499691476d65f77ef095, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:34.761222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:34.762799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:34.772608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584e1ba-79a6-4c12-9d9b-c20117aeb964 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:34.775694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 389/784] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Aug 9 19:01:34 2022] GET /identity/v3/endpoints?service_id=cbe42941a671499691476d65f77ef095&interface=public®ion_id=RegionOne => generated 194 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:34.785411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:34.818167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:34.818722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:34.819003 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:34.819270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:34.820053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:34.820379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:34.822846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:34.837206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 396/785] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:34.846544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:34.870251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:34.923487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0c88-7031-430b-b829-b361effe8bcd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:34.967398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71643c-d4c1-4bc2-b05e-69cb62c330a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:34.968012 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71643c-d4c1-4bc2-b05e-69cb62c330a2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:34.968285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71643c-d4c1-4bc2-b05e-69cb62c330a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:34.968576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71643c-d4c1-4bc2-b05e-69cb62c330a2 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:34.969180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71643c-d4c1-4bc2-b05e-69cb62c330a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:34.970534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71643c-d4c1-4bc2-b05e-69cb62c330a2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:34.981766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 390/786] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:01:34 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:35.012040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:35.030629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db7560-7fc9-42f5-bde1-05cf6b410fb8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:35.073985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dc703-2172-4336-ab07-dacb8e093de7 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:35.074162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dc703-2172-4336-ab07-dacb8e093de7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:35.074442 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dc703-2172-4336-ab07-dacb8e093de7 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:35.075828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 391/787] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:01:35.083652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d8304-6c34-4ad2-82db-f5b9c78f0e30 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:35.083936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d8304-6c34-4ad2-82db-f5b9c78f0e30 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:35.084194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d8304-6c34-4ad2-82db-f5b9c78f0e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:35.108381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-817d8304-6c34-4ad2-82db-f5b9c78f0e30 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:35.109646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-817d8304-6c34-4ad2-82db-f5b9c78f0e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:35.115950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:35.116624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:35.116899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:35.117164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:35.133046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:35.135175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:35.155099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:35.159511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:35.161547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 397/788] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:35.167193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:35.241791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 392/789] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 09 19:01:35.264939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d8304-6c34-4ad2-82db-f5b9c78f0e30 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:35.298939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c271a-167e-4e70-9a24-ae3c6cc6684f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:35.325388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:35.352794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:35.354075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:35.354351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:35.354637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:35.355435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:35.355738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:35.360404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:35.397551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d8304-6c34-4ad2-82db-f5b9c78f0e30 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:35.403074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 398/790] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Aug 9 19:01:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:35.408554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:35.414918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:35.437247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5dbf4-4450-4937-85cc-2378db0bf10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:35.438252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5dbf4-4450-4937-85cc-2378db0bf10a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:35.438722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5dbf4-4450-4937-85cc-2378db0bf10a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:35.439284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5dbf4-4450-4937-85cc-2378db0bf10a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:35.450616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5dbf4-4450-4937-85cc-2378db0bf10a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-dbe5dbf4-4450-4937-85cc-2378db0bf10a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:35.455611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:35.456557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:35.457408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:35.459949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 393/791] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:35.465537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5dbf4-4450-4937-85cc-2378db0bf10a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:35.577041 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5dbf4-4450-4937-85cc-2378db0bf10a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:35.586146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:35.616248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:35.617272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:35.617767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:35.618227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:35.619840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:35.620553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:35.622124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:35.631591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 394/792] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:35.632149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b09056-2a2a-4fd7-93a1-7730b2fc7966 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:35.639615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:35.693193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:35.694195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:35.695546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:35.696023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:35.714947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:35.716425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:35.753805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:35.754804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:35.755683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:35.755683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:35.755683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:35.760571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 399/793] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:35.767206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:35.783645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e357d23-d2f9-4688-a4ff-52eb2a788239 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:35.853973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:35.855607 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:35.856093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:35.856588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:35.878952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:35.882838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:35.883297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:35.911546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:35.912585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:35.913466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:35.913466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:35.913466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:35.918841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 395/794] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:35.927043 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:35.968493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcc27e-31b9-4ce9-8c9a-563cc4f6e57a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:35.975165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:36.053142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.054166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.054677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.055154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.056003 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:36.057261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.059195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.080625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6f3d2-37f3-4cff-bd7a-d12a268dab96 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:36.083875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 400/795] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:36.089543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:36.117741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.118275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.118557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.119440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.120092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:36.120488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.124460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.134779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 396/796] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:36.140483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:36.227993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:36.251712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322cdf3-20ae-4878-a886-64e2f057b952 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:36.266720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a0b83-e95e-485a-94b6-be5e7db6d674 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:36.285106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:36.298881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2aad0f-fcf5-49bd-9b05-532368fe9dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.300308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aad0f-fcf5-49bd-9b05-532368fe9dad None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.300308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aad0f-fcf5-49bd-9b05-532368fe9dad None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.300308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aad0f-fcf5-49bd-9b05-532368fe9dad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.311092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2aad0f-fcf5-49bd-9b05-532368fe9dad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.311709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831f041-8bba-4d03-bb95-508c4664b413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.312592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831f041-8bba-4d03-bb95-508c4664b413 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.313064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831f041-8bba-4d03-bb95-508c4664b413 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.313234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2aad0f-fcf5-49bd-9b05-532368fe9dad None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.313871 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831f041-8bba-4d03-bb95-508c4664b413 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.321144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831f041-8bba-4d03-bb95-508c4664b413 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.323077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831f041-8bba-4d03-bb95-508c4664b413 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-0d2aad0f-fcf5-49bd-9b05-532368fe9dad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:36.326907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:36.327829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:36.329033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:36.332508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 397/797] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:01:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:36.337091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc5221-9345-48d8-8145-e7b2a78f5757 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.337369 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc5221-9345-48d8-8145-e7b2a78f5757 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.337734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc5221-9345-48d8-8145-e7b2a78f5757 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.339148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 398/798] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:36.339681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc5221-9345-48d8-8145-e7b2a78f5757 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2831f041-8bba-4d03-bb95-508c4664b413 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:36.353738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:36.354926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 401/799] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cba97-3d51-4419-802e-6f3d222bc53e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.355796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cba97-3d51-4419-802e-6f3d222bc53e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.356373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cba97-3d51-4419-802e-6f3d222bc53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.376088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5a6cba97-3d51-4419-802e-6f3d222bc53e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:36.377782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5a6cba97-3d51-4419-802e-6f3d222bc53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:36.466966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc5221-9345-48d8-8145-e7b2a78f5757 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:36.476537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 402/800] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:36.477415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cba97-3d51-4419-802e-6f3d222bc53e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:36.522085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.522714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.523123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.523403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.524485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:36.524803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.526345 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.537036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:36.546273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 399/801] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Aug 9 19:01:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:36.547154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:36.606114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cba97-3d51-4419-802e-6f3d222bc53e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:36.614325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:36.644219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380d9d7-d0a1-4f44-9326-4f6df3aac01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.644773 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d9d7-d0a1-4f44-9326-4f6df3aac01f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.645054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d9d7-d0a1-4f44-9326-4f6df3aac01f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.645327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d9d7-d0a1-4f44-9326-4f6df3aac01f None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.646017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380d9d7-d0a1-4f44-9326-4f6df3aac01f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:36.646324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380d9d7-d0a1-4f44-9326-4f6df3aac01f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.647753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380d9d7-d0a1-4f44-9326-4f6df3aac01f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.653187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1df23-cb68-4426-b0df-bef2fabf66b9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:36.664222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 403/802] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:36.664222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733c22d-4bfb-40fe-924e-47e8c133a962 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.664222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733c22d-4bfb-40fe-924e-47e8c133a962 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.664222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733c22d-4bfb-40fe-924e-47e8c133a962 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.664222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 404/803] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:36.664222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733c22d-4bfb-40fe-924e-47e8c133a962 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:36.696721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.697986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.698264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.698535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.699114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.700477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:36.710284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:36.711282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:36.712108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 400/804] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Aug 9 19:01:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:36.712971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:36.781539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733c22d-4bfb-40fe-924e-47e8c133a962 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:36.785681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:36.819175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac3934-0e9f-4a4e-a370-b24e039634d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.819799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3934-0e9f-4a4e-a370-b24e039634d5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.820081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3934-0e9f-4a4e-a370-b24e039634d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.820358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac3934-0e9f-4a4e-a370-b24e039634d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.830027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d93213-243e-4084-9928-ae2299a17714 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:36.832455 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac3934-0e9f-4a4e-a370-b24e039634d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.834016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac3934-0e9f-4a4e-a370-b24e039634d5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-34ac3934-0e9f-4a4e-a370-b24e039634d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:36.848727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:36.849885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:36.850998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:36.850998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:36.850998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:36.851689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 405/805] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:36.852986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7b202-b3be-4f43-82a4-d69a75a951b4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.853431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7b202-b3be-4f43-82a4-d69a75a951b4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.853775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7b202-b3be-4f43-82a4-d69a75a951b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.870870 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43775ff-42c8-4392-9991-17893b62cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:36.871540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43775ff-42c8-4392-9991-17893b62cac5 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:36.871818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43775ff-42c8-4392-9991-17893b62cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:36.872143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43775ff-42c8-4392-9991-17893b62cac5 None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/141d2c2f92c842feb6155aae1e9edfd2/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:36.873102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7fb7b202-b3be-4f43-82a4-d69a75a951b4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:36.874097 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43775ff-42c8-4392-9991-17893b62cac5 None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=141d2c2f92c842feb6155aae1e9edfd2, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:36.874423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7fb7b202-b3be-4f43-82a4-d69a75a951b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:36.875510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43775ff-42c8-4392-9991-17893b62cac5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:36.889126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:36.892730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 401/806] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:36 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/141d2c2f92c842feb6155aae1e9edfd2/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:36.901008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43775ff-42c8-4392-9991-17893b62cac5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:36.950199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 406/807] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:36.951274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7b202-b3be-4f43-82a4-d69a75a951b4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.061017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43775ff-42c8-4392-9991-17893b62cac5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.088641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:37.095318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7b202-b3be-4f43-82a4-d69a75a951b4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.105016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.106407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.106407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.106407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.106801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:37.107062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:37.108596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:37.118468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 402/808] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Aug 9 19:01:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:37.120775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.142334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.142944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.143216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.143486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.144137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:37.144463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:37.145889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:37.159480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 407/809] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:37.161128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.176432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:37.246177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2b01b-3097-4876-ace9-7cc6e6993c2b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.285692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2847e0f-483f-4006-a31a-167f0fc2eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.286468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2847e0f-483f-4006-a31a-167f0fc2eb06 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.286743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2847e0f-483f-4006-a31a-167f0fc2eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.287104 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2847e0f-483f-4006-a31a-167f0fc2eb06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.290175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b885e3-aa08-4a20-aaf5-8446876e0525 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.299418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2847e0f-483f-4006-a31a-167f0fc2eb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d2847e0f-483f-4006-a31a-167f0fc2eb06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:37.304960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:37.306009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 403/810] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:37.307436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2847e0f-483f-4006-a31a-167f0fc2eb06 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.335747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.336460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.336807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.337311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.339042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:37.339415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:37.340922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:37.346061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 408/811] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Aug 9 19:01:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:37.346950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.360079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:37.419376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2847e0f-483f-4006-a31a-167f0fc2eb06 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.443901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:37.465474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.466121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.466399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.466671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.473849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:37.475286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:37.475904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562a2bc-0070-4b43-b32d-9ca53aa801f4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:37.488212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:37.489198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 404/812] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:37.490072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.521456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c3f65-7f55-497b-a4f2-ed74949044f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.523652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c3f65-7f55-497b-a4f2-ed74949044f3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.523652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c3f65-7f55-497b-a4f2-ed74949044f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.523652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c3f65-7f55-497b-a4f2-ed74949044f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.523652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c3f65-7f55-497b-a4f2-ed74949044f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:37.523652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c3f65-7f55-497b-a4f2-ed74949044f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:37.524973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c3f65-7f55-497b-a4f2-ed74949044f3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:37.533956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 409/813] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:37.535119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88114-8c37-4497-81a3-08478e55a0f6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.535403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88114-8c37-4497-81a3-08478e55a0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.535676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88114-8c37-4497-81a3-08478e55a0f6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.544483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 410/814] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:37.544483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88114-8c37-4497-81a3-08478e55a0f6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.607622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f4f2-3014-49c9-afc9-d0b081100120 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.616050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:37.644697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4f9b0-3608-4d14-9dc2-4d860203e007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.645304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4f9b0-3608-4d14-9dc2-4d860203e007 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.645581 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4f9b0-3608-4d14-9dc2-4d860203e007 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.645849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4f9b0-3608-4d14-9dc2-4d860203e007 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.646503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4f9b0-3608-4d14-9dc2-4d860203e007 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:37.647857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4f9b0-3608-4d14-9dc2-4d860203e007 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:37.656576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88114-8c37-4497-81a3-08478e55a0f6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.680565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 405/815] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:01:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:37.680565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4f9b0-3608-4d14-9dc2-4d860203e007 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.710151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:37.717835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ab71a-30a4-4acd-ac7c-295bcf14ca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.718898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab71a-30a4-4acd-ac7c-295bcf14ca7a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.719684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab71a-30a4-4acd-ac7c-295bcf14ca7a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.719959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab71a-30a4-4acd-ac7c-295bcf14ca7a None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.720977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ab71a-30a4-4acd-ac7c-295bcf14ca7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:37.721286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ab71a-30a4-4acd-ac7c-295bcf14ca7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:37.723101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ab71a-30a4-4acd-ac7c-295bcf14ca7a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:37.740985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 411/816] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:37.742761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a83eb6-3fa2-4497-b673-df092b09e8ea None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.743046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a83eb6-3fa2-4497-b673-df092b09e8ea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.743315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a83eb6-3fa2-4497-b673-df092b09e8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.769539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-17a83eb6-3fa2-4497-b673-df092b09e8ea None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:37.770762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-17a83eb6-3fa2-4497-b673-df092b09e8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:37.815383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4f9b0-3608-4d14-9dc2-4d860203e007 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:37.871682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 412/817] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:37.872413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a83eb6-3fa2-4497-b673-df092b09e8ea None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.877238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:37.877882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:37.878159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:37.878400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:37.896387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:37.898207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:37.914271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:37.915241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:37.917037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:37.917037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:37.917037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:37.917037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 406/818] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:37.917037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:37.932902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:38.014497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a83eb6-3fa2-4497-b673-df092b09e8ea None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:38.036550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:38.040505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afe521-be75-4bb3-ba70-5a769bf91930 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:38.053875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb85958-662f-458f-945e-36fc890321bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:38.054613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb85958-662f-458f-945e-36fc890321bd None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:38.055012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb85958-662f-458f-945e-36fc890321bd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:38.055357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb85958-662f-458f-945e-36fc890321bd None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/87b647f3da7246b0b530a65c500d5064/roles/72882169372f4db5bd4bf8c9c399b812` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:38.076159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb85958-662f-458f-945e-36fc890321bd None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=87b647f3da7246b0b530a65c500d5064, role_id=72882169372f4db5bd4bf8c9c399b812)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:38.078125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb85958-662f-458f-945e-36fc890321bd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:38.083484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:38.084168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:38.084575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:38.084759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] PATH_INFO: `/v3/services/network` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:38.085299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:38.086728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:38.096225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:38.097228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 19:01:38.099099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:38.100993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 413/819] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:37 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/87b647f3da7246b0b530a65c500d5064/roles/72882169372f4db5bd4bf8c9c399b812 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:38.100993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb85958-662f-458f-945e-36fc890321bd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:38.101282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 407/820] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:37 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:38.108539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:38.215775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:38.246673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64155795-13ab-4a5e-8835-fdfda8dff38b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:38.281169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb85958-662f-458f-945e-36fc890321bd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:38.300487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:38.300487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:38.300487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:38.300487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:38.300976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:38.300976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:38.300976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:38.308054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 408/821] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:38.315464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:38.362218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:38.362832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:38.363152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:38.363428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:38.364073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:38.364371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:38.366241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:38.390879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 414/822] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:38.401174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:38.444335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e7b7c-10d5-4675-9bbb-91d04ea005a5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:38.481868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:38.482392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:38.482668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:38.482938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:38.483564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] RBAC: Adding query filter params (type=network) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:38.483917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:38.485357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:38.486014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:38.491753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 409/823] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:38.499394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:38.509975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:38.614763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69154dbc-9de6-48c1-a09a-f15984002a06 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:38.651936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201b1c-e122-4a12-902d-53b8dbdbae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:38.652577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201b1c-e122-4a12-902d-53b8dbdbae32 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:38.652911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201b1c-e122-4a12-902d-53b8dbdbae32 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:38.653204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201b1c-e122-4a12-902d-53b8dbdbae32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:38.653847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201b1c-e122-4a12-902d-53b8dbdbae32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:38.655254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201b1c-e122-4a12-902d-53b8dbdbae32 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:38.674732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5968332-ae77-46ac-854d-dc3b5f430884 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:38.699771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 410/824] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:01:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:38.720537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:38.721570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:38.722054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:38.722525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:38.733072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:38.734925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:38.748172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:38.749738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:01:38.750662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:38.753440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 415/825] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:01:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:38.759543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201b1c-e122-4a12-902d-53b8dbdbae32 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:38.844547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:38.899866 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201b1c-e122-4a12-902d-53b8dbdbae32 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:38.937805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523cc8d-9c7d-4521-83d1-e172916a31ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:38.938416 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523cc8d-9c7d-4521-83d1-e172916a31ac None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:38.938694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523cc8d-9c7d-4521-83d1-e172916a31ac None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:38.938968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523cc8d-9c7d-4521-83d1-e172916a31ac None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:38.939758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523cc8d-9c7d-4521-83d1-e172916a31ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:38.940067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523cc8d-9c7d-4521-83d1-e172916a31ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:38.941510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523cc8d-9c7d-4521-83d1-e172916a31ac None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:38.963065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 411/826] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:01:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:38.971836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.073748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:39.091896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef832-a35e-41c8-9b75-69099e100212 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:39.131131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea68229-514c-4c54-bc8d-0a07568446dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:39.131743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea68229-514c-4c54-bc8d-0a07568446dd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.132022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea68229-514c-4c54-bc8d-0a07568446dd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.132297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea68229-514c-4c54-bc8d-0a07568446dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.209944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea68229-514c-4c54-bc8d-0a07568446dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:39.211405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea68229-514c-4c54-bc8d-0a07568446dd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:39.220545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5f63-8016-4142-a27c-112836b1758c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.220869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5f63-8016-4142-a27c-112836b1758c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.221107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5f63-8016-4142-a27c-112836b1758c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.223815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 412/827] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:39.227852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4d33e-1879-4a5b-8441-7442f84bb04f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.228822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4d33e-1879-4a5b-8441-7442f84bb04f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.229222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4d33e-1879-4a5b-8441-7442f84bb04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.251712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-62a4d33e-1879-4a5b-8441-7442f84bb04f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:39.253011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-62a4d33e-1879-4a5b-8441-7442f84bb04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:39.305623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:39.328869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 416/828] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:01:39.334427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea68229-514c-4c54-bc8d-0a07568446dd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.352149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 413/829] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:01:39.371569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4d33e-1879-4a5b-8441-7442f84bb04f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.437753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:39.459251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea68229-514c-4c54-bc8d-0a07568446dd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:39.498596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4d33e-1879-4a5b-8441-7442f84bb04f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:39.506891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:39.507807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.508758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.509263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.523607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:39.526760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:39.535562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60762a91-f349-4c2b-bbaa-7eb86134fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:39.536202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60762a91-f349-4c2b-bbaa-7eb86134fb31 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.536511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60762a91-f349-4c2b-bbaa-7eb86134fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.537237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60762a91-f349-4c2b-bbaa-7eb86134fb31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:39.547869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:39.548879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:39.549796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:39.549796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:39.549796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:39.555259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 417/830] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:39.555426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60762a91-f349-4c2b-bbaa-7eb86134fb31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:39.557661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-60762a91-f349-4c2b-bbaa-7eb86134fb31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:39.560530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:39.561516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:01:39.564413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:39.566898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 414/831] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:39.569098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60762a91-f349-4c2b-bbaa-7eb86134fb31 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.622891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:39.690549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1bc7f-3d04-47df-b790-10c0ec17f738 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:39.703560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60762a91-f349-4c2b-bbaa-7eb86134fb31 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:39.720126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:39.733507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:39.734119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.734407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.734686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.735333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:39.735635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:39.737593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:39.760030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 418/832] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:39.761200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:39.761824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.762105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.762381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.763058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:39.763363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:39.764762 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:39.765473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.778341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 415/833] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:39.779646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.878735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:39.883113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b155a-d814-4328-af8b-1c11170aba5b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:39.883430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9251a0-c6d4-40ed-8f5c-2bd17c73c3be None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:39.923449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:39.924969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.925469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.925962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.930027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:39.930535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:39.930909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:39.931409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:39.936924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:39.938800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:39.944017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:39.945480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:39.952888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:39.953779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:39.954671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:39.954671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:39.954671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:39.954671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:39.954671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:39.954671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:39.954671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:39.954671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:39.961432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:39.962373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:39.964320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:39.964320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:39.964320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:39.965165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 419/835] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:39.966319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.967185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 416/835] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:39.968452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:39.995527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:40.079240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82087a-6925-414e-b009-509392fbbe6c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:40.084901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e05a8-0c5a-4a83-89ad-3e7809e12ecb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:40.115788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:40.116641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.116844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.117073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.117718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:40.118021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:40.119954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:40.126929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:40.127891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.128374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.128945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.129884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:40.130427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:40.132056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:40.137454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 417/836] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:40.138798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:40.153302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 420/837] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:40.154649 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:40.175993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:40.233632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:40.252208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd7988-3e00-484e-a800-83bbdc3111c1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:40.266595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565d51a-5a3b-4362-ae25-bfae6ad4e621 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:40.297009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aed130-a079-4555-926d-f06324622fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:40.297768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aed130-a079-4555-926d-f06324622fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.298049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aed130-a079-4555-926d-f06324622fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.298317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aed130-a079-4555-926d-f06324622fa0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.306014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129ebfe-0589-40e1-8153-70c119bdd570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:40.306225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aed130-a079-4555-926d-f06324622fa0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:40.306755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ebfe-0589-40e1-8153-70c119bdd570 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.307115 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ebfe-0589-40e1-8153-70c119bdd570 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.307315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ebfe-0589-40e1-8153-70c119bdd570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.308004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aed130-a079-4555-926d-f06324622fa0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:40.308099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129ebfe-0589-40e1-8153-70c119bdd570 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=48410d57fe0d489cbb07cfca58ca7b30) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:40.308404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129ebfe-0589-40e1-8153-70c119bdd570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:40.310135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129ebfe-0589-40e1-8153-70c119bdd570 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:40.324468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 421/838] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:40 2022] GET /identity/v3/role_assignments?user.id=48410d57fe0d489cbb07cfca58ca7b30&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:40.324468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611e94a-5efa-4e35-a24f-509b036e88f9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.324468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611e94a-5efa-4e35-a24f-509b036e88f9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.324468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611e94a-5efa-4e35-a24f-509b036e88f9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.324468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 422/839] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-00aed130-a079-4555-926d-f06324622fa0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:40.324939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:40.326003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:40.327129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 418/840] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:01:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:40.327838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611e94a-5efa-4e35-a24f-509b036e88f9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:40.333918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda0f27-e41c-4adc-8650-e7d5914f30b9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.334195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda0f27-e41c-4adc-8650-e7d5914f30b9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.334469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda0f27-e41c-4adc-8650-e7d5914f30b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.358959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2fda0f27-e41c-4adc-8650-e7d5914f30b9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:40.360169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2fda0f27-e41c-4adc-8650-e7d5914f30b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:40.437687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:40.461344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611e94a-5efa-4e35-a24f-509b036e88f9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:40.463512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 419/841] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:40.486777 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda0f27-e41c-4adc-8650-e7d5914f30b9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:40.510968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e7a5b-c94f-4d18-b4c2-eac66940e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:40.511595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7a5b-c94f-4d18-b4c2-eac66940e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.511872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7a5b-c94f-4d18-b4c2-eac66940e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.512146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7a5b-c94f-4d18-b4c2-eac66940e7e4 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.512966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e7a5b-c94f-4d18-b4c2-eac66940e7e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:40.513276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e7a5b-c94f-4d18-b4c2-eac66940e7e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:40.514706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e7a5b-c94f-4d18-b4c2-eac66940e7e4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:40.535026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 423/842] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Aug 9 19:01:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:40.538539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf1dc7-3a59-4b19-b683-ea2c8437f133 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.538813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf1dc7-3a59-4b19-b683-ea2c8437f133 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.539087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf1dc7-3a59-4b19-b683-ea2c8437f133 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.542052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 424/843] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:40.542264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf1dc7-3a59-4b19-b683-ea2c8437f133 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:40.586928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:40.612045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda0f27-e41c-4adc-8650-e7d5914f30b9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:40.652067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff6b70-8e4a-4d6b-8bd0-0a94bd40e5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:40.652716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6b70-8e4a-4d6b-8bd0-0a94bd40e5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.652989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6b70-8e4a-4d6b-8bd0-0a94bd40e5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.653256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6b70-8e4a-4d6b-8bd0-0a94bd40e5e9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.653828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff6b70-8e4a-4d6b-8bd0-0a94bd40e5e9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:40.654435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf1dc7-3a59-4b19-b683-ea2c8437f133 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:40.655180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff6b70-8e4a-4d6b-8bd0-0a94bd40e5e9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-1bff6b70-8e4a-4d6b-8bd0-0a94bd40e5e9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:40.664899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:40.672024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:40.673272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 420/844] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:40.674364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f065b-7540-481f-a86e-e86133cbff93 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.674364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f065b-7540-481f-a86e-e86133cbff93 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.674364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f065b-7540-481f-a86e-e86133cbff93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.691719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5d3f065b-7540-481f-a86e-e86133cbff93 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:40.692605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180da08-e70f-4aa6-9cad-61c9f4910511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:40.693075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5d3f065b-7540-481f-a86e-e86133cbff93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:40.693359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180da08-e70f-4aa6-9cad-61c9f4910511 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.693701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180da08-e70f-4aa6-9cad-61c9f4910511 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.694035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180da08-e70f-4aa6-9cad-61c9f4910511 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.705342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:40.765577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180da08-e70f-4aa6-9cad-61c9f4910511 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:40.766419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 421/845] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:40.767252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f065b-7540-481f-a86e-e86133cbff93 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:40.777334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180da08-e70f-4aa6-9cad-61c9f4910511 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:40.837019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:40.875589 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f065b-7540-481f-a86e-e86133cbff93 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:40.892083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 425/846] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:01:40.893088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180da08-e70f-4aa6-9cad-61c9f4910511 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:40.917693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:40.918246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:40.918521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:40.918911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:40.919643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:40.919942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:40.921333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:40.926148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 422/847] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:40.927022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:40.943402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:41.006940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180da08-e70f-4aa6-9cad-61c9f4910511 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.032711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedda99-82cd-420b-a40c-74c748b35dc8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.042414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0cf3-590e-4a72-a08e-8332f7df0428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.043040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0cf3-590e-4a72-a08e-8332f7df0428 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.043310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0cf3-590e-4a72-a08e-8332f7df0428 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.043582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0cf3-590e-4a72-a08e-8332f7df0428 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.054331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e0cf3-590e-4a72-a08e-8332f7df0428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1e7e0cf3-590e-4a72-a08e-8332f7df0428 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:41.060274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:41.061388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:41.062230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 426/848] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:01:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:41.062913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0cf3-590e-4a72-a08e-8332f7df0428 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.070972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:41.078694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.079424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.079703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.079969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.091132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.092884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:41.108941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:41.114651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:41.115823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:41.115823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:41.115823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:41.115823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 423/849] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:41.115823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.171068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:41.174046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e0cf3-590e-4a72-a08e-8332f7df0428 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.209212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.209896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.210241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.210575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.211260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:41.211628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.213693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.228657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 427/850] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:41.228657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.234627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703962ea-5822-4408-b6a8-2b6a34950549 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.275923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.276702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.277161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.277488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.278295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:41.278668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.280342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.295810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 424/851] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:41.297705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.355083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:41.369904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6187e-0760-4217-8f36-d144f233efb1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.422995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.424064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.424551 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.424991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.426478 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:41.426967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.429332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.431088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152a9a9-425b-4be3-bf54-d7a54fbd2ec8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.445549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 428/852] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:41.451921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.481261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.481656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.481941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.482216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.485383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:41.496674 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.498243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:41.515435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:41.516467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:41.517352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:41.517352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:41.517352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:41.517352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 425/853] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:41.522498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.581537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dc3c1-47e5-4307-90ff-1f4c73f1d222 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.630680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.631942 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.632887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.633621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.638433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:41.642651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22461d-ad66-449a-acc9-1d896cd2f01c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.644629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.647046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:41.662882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:41.663949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:41.664941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:41.664941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:41.664941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:41.664941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:41.664941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:41.664941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:41.664941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:41.664941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:41.670658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 429/854] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:41.671511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.687454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.688211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.688972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.689253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.690156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:41.690466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.692205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.710311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 426/855] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:41.716682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.777067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:41.799337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33a46b-bbf6-42da-9310-5894b21023e4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.844824 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.846001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.846432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.847085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.848322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:41.848790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.850866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.858892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae35fe5-09d6-4e20-b38f-b55880b48245 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:41.873175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 430/856] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:41.879854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.906292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:41.906918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:41.907189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:41.907456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:41.915335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:41.917400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:41.928923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:41.929916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:41.930849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:41.935094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 427/857] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:41.936071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:41.955732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:42.020273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d0c44-ac54-48a2-a4ba-947e96283b6a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:42.057310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:42.058359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.058359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.058566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.059232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=141d2c2f92c842feb6155aae1e9edfd2) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:42.059842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:42.061284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:42.072701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 431/858] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/role_assignments?user.id=141d2c2f92c842feb6155aae1e9edfd2&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:42.078459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4407387-6f9f-4312-acc0-b1a32edabd2c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:42.117766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7847d1-f53b-456e-b08a-f2b87cd84772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:42.118379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7847d1-f53b-456e-b08a-f2b87cd84772 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.118828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7847d1-f53b-456e-b08a-f2b87cd84772 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.119237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7847d1-f53b-456e-b08a-f2b87cd84772 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.120008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7847d1-f53b-456e-b08a-f2b87cd84772 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:42.120362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7847d1-f53b-456e-b08a-f2b87cd84772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:42.121848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7847d1-f53b-456e-b08a-f2b87cd84772 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:42.143050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 428/859] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Aug 9 19:01:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:42.148727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:42.162405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:42.270010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15b06a-279a-4feb-bebf-8da3cf8953e7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:42.311033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d88f35-5f9c-4f99-a47f-fa6ba3885e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:42.311893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d88f35-5f9c-4f99-a47f-fa6ba3885e50 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.312321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d88f35-5f9c-4f99-a47f-fa6ba3885e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.312824 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d88f35-5f9c-4f99-a47f-fa6ba3885e50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.387426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d88f35-5f9c-4f99-a47f-fa6ba3885e50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:42.389161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d88f35-5f9c-4f99-a47f-fa6ba3885e50 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:42.418784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:42.476608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79882583-7caf-4f16-aeb0-999827d54c94 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.476924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79882583-7caf-4f16-aeb0-999827d54c94 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.477167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79882583-7caf-4f16-aeb0-999827d54c94 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.483155 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 429/860] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:42.484733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690a5b-c24e-4790-8e76-95acde06a0dd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.485016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690a5b-c24e-4790-8e76-95acde06a0dd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.485289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690a5b-c24e-4790-8e76-95acde06a0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.502078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 432/861] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:01:42.504145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-28690a5b-c24e-4790-8e76-95acde06a0dd None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:42.505375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-28690a5b-c24e-4790-8e76-95acde06a0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:42.507152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d88f35-5f9c-4f99-a47f-fa6ba3885e50 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:42.580824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 430/862] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:01:42.605003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28690a5b-c24e-4790-8e76-95acde06a0dd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:42.614619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:42.648875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d88f35-5f9c-4f99-a47f-fa6ba3885e50 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:42.663295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:42.698532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:42.699171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.699456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.699722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.713574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:42.715486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:42.727174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28690a5b-c24e-4790-8e76-95acde06a0dd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:42.734588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:42.735528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:42.737079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:42.737079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:42.737079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:42.737079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 433/863] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:42.741133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:42.768503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:42.769511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.769988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.770494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.771309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:42.772972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:42.782604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:42.783555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:42.784450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:42.792396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 431/864] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Aug 9 19:01:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:42.793879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:42.865186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492337c-f3d8-458c-890c-442eb46f0a06 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:42.886277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:42.913154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:42.913766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.914065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.914579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.915232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:42.916132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:42.918829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:42.924360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c4a30-3ade-4199-bca2-4cf0dda7f63b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:42.939934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 434/865] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:42.942202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:42.967133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:42.967895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:42.968281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:42.968675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:42.969410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:42.969872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:42.971315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:42.975115 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:42.981272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 432/866] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:42.983976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:43.062076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046e214-3b1b-4515-b100-1f7b230d383b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:43.098312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:43.099286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:43.099668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:43.100033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:43.108184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:43.109899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:43.117177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750a4f0-3748-4c09-b5ff-0541787965b1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:43.122685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:43.123765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:43.124893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:43.124893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:43.124893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:43.124893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:43.124893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:43.124893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:43.124893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:43.124893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:43.131047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 435/867] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:43.132275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:43.173938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:43.174817 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:43.175635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:43.176065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:43.177250 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:43.177911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:43.179747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:43.182407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:43.188036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 433/868] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:43.232854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:43.253175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eff1e3-b0cd-42ba-9768-460d33cd1257 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:43.295573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd99ce5-7b63-4374-8ceb-a85e3b09f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:43.296915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd99ce5-7b63-4374-8ceb-a85e3b09f246 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:43.297368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd99ce5-7b63-4374-8ceb-a85e3b09f246 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:43.297800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd99ce5-7b63-4374-8ceb-a85e3b09f246 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:43.298669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd99ce5-7b63-4374-8ceb-a85e3b09f246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:43.299123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd99ce5-7b63-4374-8ceb-a85e3b09f246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:43.300816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd99ce5-7b63-4374-8ceb-a85e3b09f246 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:43.320833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 436/869] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:43.324195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:43.354727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750436-e87d-40c1-87a6-498fd717a8d1 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:43.355280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750436-e87d-40c1-87a6-498fd717a8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:43.355718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750436-e87d-40c1-87a6-498fd717a8d1 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:43.357454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 437/870] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:43.365466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44079ffd-232e-42c6-88a2-3e3704e3a5d4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:43.365832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44079ffd-232e-42c6-88a2-3e3704e3a5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:43.366182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44079ffd-232e-42c6-88a2-3e3704e3a5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:43.390732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-44079ffd-232e-42c6-88a2-3e3704e3a5d4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:43.391112 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-44079ffd-232e-42c6-88a2-3e3704e3a5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:43.474064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a6476-941d-4abd-9503-3e4026f939cc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:43.484327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 438/871] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:43.507191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44079ffd-232e-42c6-88a2-3e3704e3a5d4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:43.516504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13167fdf-f951-49ea-8986-91dbfca9090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:43.517485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13167fdf-f951-49ea-8986-91dbfca9090b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:43.517961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13167fdf-f951-49ea-8986-91dbfca9090b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:43.518444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13167fdf-f951-49ea-8986-91dbfca9090b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:43.519413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13167fdf-f951-49ea-8986-91dbfca9090b None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=87b647f3da7246b0b530a65c500d5064) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:43.519964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13167fdf-f951-49ea-8986-91dbfca9090b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:43.521618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13167fdf-f951-49ea-8986-91dbfca9090b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:43.528631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 434/872] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:43 2022] GET /identity/v3/role_assignments?user.id=87b647f3da7246b0b530a65c500d5064&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:43.591424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:43.627815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44079ffd-232e-42c6-88a2-3e3704e3a5d4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:43.667334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfa352-c2ff-48be-961f-42f41e7339ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:43.667951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa352-c2ff-48be-961f-42f41e7339ae None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:43.668228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa352-c2ff-48be-961f-42f41e7339ae None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:43.668598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa352-c2ff-48be-961f-42f41e7339ae None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:43.669250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfa352-c2ff-48be-961f-42f41e7339ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:43.670640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfa352-c2ff-48be-961f-42f41e7339ae None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:43.689342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 439/873] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:01:43 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:44.267950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a84ce-dba0-4e6b-8027-6b6ca4082418 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:44.268488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a84ce-dba0-4e6b-8027-6b6ca4082418 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:44.268758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a84ce-dba0-4e6b-8027-6b6ca4082418 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:44.270465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 435/874] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:44.275913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee65a-815f-4f52-a54d-0c121e704a99 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:44.276309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee65a-815f-4f52-a54d-0c121e704a99 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:44.276712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee65a-815f-4f52-a54d-0c121e704a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:44.296941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5b2ee65a-815f-4f52-a54d-0c121e704a99 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:44.298128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5b2ee65a-815f-4f52-a54d-0c121e704a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:44.382473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 440/875] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:44.404035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a84ce-dba0-4e6b-8027-6b6ca4082418 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:44.471655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:44.517422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a84ce-dba0-4e6b-8027-6b6ca4082418 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:44.552086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:44.552677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:44.552955 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:44.553229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:44.553805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:44.555209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:44.564736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:44.565692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:44.566673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 436/876] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:44.571837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee65a-815f-4f52-a54d-0c121e704a99 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:44.646221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:44.681743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee65a-815f-4f52-a54d-0c121e704a99 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:44.717526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9b85-ac00-46cc-ae42-9bc6afa9dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:44.718137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9b85-ac00-46cc-ae42-9bc6afa9dbde None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:44.718414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9b85-ac00-46cc-ae42-9bc6afa9dbde None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:44.718686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9b85-ac00-46cc-ae42-9bc6afa9dbde None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:44.719327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b9b85-ac00-46cc-ae42-9bc6afa9dbde None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:44.719627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b9b85-ac00-46cc-ae42-9bc6afa9dbde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:44.721029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b9b85-ac00-46cc-ae42-9bc6afa9dbde None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:44.726084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 441/877] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:44.732035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:44.836702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c5b4b-336a-4d1a-90f3-624e42bebc70 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:44.866115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:44.876386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ce7ad-ec2d-4612-8c7e-6cd6c5cc8d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:44.877239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ce7ad-ec2d-4612-8c7e-6cd6c5cc8d99 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:44.877517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ce7ad-ec2d-4612-8c7e-6cd6c5cc8d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:44.877789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ce7ad-ec2d-4612-8c7e-6cd6c5cc8d99 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:44.878430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ce7ad-ec2d-4612-8c7e-6cd6c5cc8d99 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:44.878776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ce7ad-ec2d-4612-8c7e-6cd6c5cc8d99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:44.880139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ce7ad-ec2d-4612-8c7e-6cd6c5cc8d99 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:44.885886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 437/878] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:45.305039 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69776f0-7acc-41a1-a536-127086077bb2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:45.306779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69776f0-7acc-41a1-a536-127086077bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:45.307317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69776f0-7acc-41a1-a536-127086077bb2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:45.309222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 442/879] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:45.314973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fe991-7de5-4a81-aaa0-a131f3e58220 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:45.315397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fe991-7de5-4a81-aaa0-a131f3e58220 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:45.315580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fe991-7de5-4a81-aaa0-a131f3e58220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:45.334044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-395fe991-7de5-4a81-aaa0-a131f3e58220 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:45.335230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-395fe991-7de5-4a81-aaa0-a131f3e58220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:45.410257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 438/880] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:45.433329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69776f0-7acc-41a1-a536-127086077bb2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:45.537449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69776f0-7acc-41a1-a536-127086077bb2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:45.565752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:45.580714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802ddd2-0084-47c6-9faa-1b6d9376d62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:45.581562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802ddd2-0084-47c6-9faa-1b6d9376d62b None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:45.581902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802ddd2-0084-47c6-9faa-1b6d9376d62b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:45.582399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802ddd2-0084-47c6-9faa-1b6d9376d62b None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:45.583056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802ddd2-0084-47c6-9faa-1b6d9376d62b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:45.584460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802ddd2-0084-47c6-9faa-1b6d9376d62b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:45.604104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 443/881] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:01:45 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:45.824853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7407c-50eb-41f1-9acd-6a91e74f65fb None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:45.825206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7407c-50eb-41f1-9acd-6a91e74f65fb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:45.825312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7407c-50eb-41f1-9acd-6a91e74f65fb None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:45.826923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 439/882] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:45.832192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32ffe-ba5b-434b-8adf-295ba26d8c5a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:45.832513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32ffe-ba5b-434b-8adf-295ba26d8c5a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:45.832794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32ffe-ba5b-434b-8adf-295ba26d8c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:45.851777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-77e32ffe-ba5b-434b-8adf-295ba26d8c5a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:45.853027 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-77e32ffe-ba5b-434b-8adf-295ba26d8c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:45.905082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624ae99-8f0d-458d-996e-dc2c6be05e7e None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:45.905415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624ae99-8f0d-458d-996e-dc2c6be05e7e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:45.905622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624ae99-8f0d-458d-996e-dc2c6be05e7e None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:45.907182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 440/883] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:45.912162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3aea-fbff-411d-b1af-4110dfb87add None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:45.912463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3aea-fbff-411d-b1af-4110dfb87add None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:45.912751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3aea-fbff-411d-b1af-4110dfb87add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:45.932255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 444/884] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:45.934068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2abd3aea-fbff-411d-b1af-4110dfb87add None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:45.935524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2abd3aea-fbff-411d-b1af-4110dfb87add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:45.962821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e32ffe-ba5b-434b-8adf-295ba26d8c5a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.013419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 441/885] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:01:46.035393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd3aea-fbff-411d-b1af-4110dfb87add None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.090269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:46.092045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e32ffe-ba5b-434b-8adf-295ba26d8c5a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.100978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:46.134571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62c448-a30d-4359-8b80-3b810f3f1a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.135846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62c448-a30d-4359-8b80-3b810f3f1a1f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.136328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62c448-a30d-4359-8b80-3b810f3f1a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.136853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62c448-a30d-4359-8b80-3b810f3f1a1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.144117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd3aea-fbff-411d-b1af-4110dfb87add None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.151063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62c448-a30d-4359-8b80-3b810f3f1a1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ed62c448-a30d-4359-8b80-3b810f3f1a1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:46.157869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:46.158784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:01:46.159623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:46.162534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 445/886] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Aug 9 19:01:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:46.172418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62c448-a30d-4359-8b80-3b810f3f1a1f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.187961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.188803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.189236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.189610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.190296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.191831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:46.201277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:46.202502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:46.203372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:46.208461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 442/887] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:46.213782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.284001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62c448-a30d-4359-8b80-3b810f3f1a1f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.321732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.322681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.322968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.323246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.323918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:46.324222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.324428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34789196-6428-4b4f-92a8-1081e805fdb6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.325716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.339749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:46.339944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 446/888] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:46.341535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.371631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.372380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.372767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.373114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.374151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:46.374522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.376451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.379813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:46.386121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 443/889] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:46.391867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.467280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c684631-b120-493f-a53c-ea6d1ba1e8be None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.507954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.509014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.510186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.510872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.523510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.526500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.535158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff834e38-2d33-4f39-8c29-856f2b6e60fe None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:46.542190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:46.543125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:46.543994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:46.543994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:46.543994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:46.546245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 447/890] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:46.551472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.593394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.594012 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.594368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.594706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.595406 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:46.595775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.597217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.602698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 444/891] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:46.608004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.657892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:46.672083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:46.698863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dfb26-ef41-48df-941c-f426c8b09416 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.736826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.737537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.737873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.738205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.738917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:46.739282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.740306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8383d-c540-4dcb-84de-d24215ef189d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.740793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.758657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 448/892] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:46.759776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.784302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.785151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.786242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.786450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.787153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] RBAC: Adding query filter params (interface=public, service_id=c27311ae6d564e7e93df472db46e1f1a, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:46.787461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.788836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.795299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 445/893] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/endpoints?service_id=c27311ae6d564e7e93df472db46e1f1a&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:46.801303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.896044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868c371-4b27-43e7-a77a-cd3fe11d4298 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.922529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e83c7-46f4-4d46-8758-7ff69e5dc5d8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:46.933380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:46.935452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bb63c-73cf-4c4b-afc0-44b812895557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.936417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bb63c-73cf-4c4b-afc0-44b812895557 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.936823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bb63c-73cf-4c4b-afc0-44b812895557 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.937198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bb63c-73cf-4c4b-afc0-44b812895557 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.945139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bb63c-73cf-4c4b-afc0-44b812895557 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.946662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bb63c-73cf-4c4b-afc0-44b812895557 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.950154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-af9bb63c-73cf-4c4b-afc0-44b812895557 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:46.960630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:46.961808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:46.963483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:46.964079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 449/894] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:46.965108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004999e5-0e95-4440-a99d-914d12b6e43a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.965390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004999e5-0e95-4440-a99d-914d12b6e43a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.965663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004999e5-0e95-4440-a99d-914d12b6e43a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.967939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 450/895] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:46.970637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004999e5-0e95-4440-a99d-914d12b6e43a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:46.972725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15e3fa-4b82-477a-a99b-ccc8f2114d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:46.973337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e3fa-4b82-477a-a99b-ccc8f2114d59 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.973620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e3fa-4b82-477a-a99b-ccc8f2114d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.973892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e3fa-4b82-477a-a99b-ccc8f2114d59 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:46.974709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15e3fa-4b82-477a-a99b-ccc8f2114d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:46.976076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15e3fa-4b82-477a-a99b-ccc8f2114d59 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:46.982902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 446/896] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:01:46.984074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff09b-282d-4fdb-b0b4-37fd866034e5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:46.984356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff09b-282d-4fdb-b0b4-37fd866034e5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:46.984735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff09b-282d-4fdb-b0b4-37fd866034e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:47.006660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-855ff09b-282d-4fdb-b0b4-37fd866034e5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:47.008706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-855ff09b-282d-4fdb-b0b4-37fd866034e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:47.100463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 447/897] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:47.115930 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004999e5-0e95-4440-a99d-914d12b6e43a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:47.123809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ff09b-282d-4fdb-b0b4-37fd866034e5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:47.162094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:47.162851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:47.163132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:47.163485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:47.164287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:47.164621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:47.166359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:47.190837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 451/898] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Aug 9 19:01:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:47.197170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:47.237507 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:47.241246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ff09b-282d-4fdb-b0b4-37fd866034e5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:47.273212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:47.289091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f044f27-b758-4d57-9212-d5e06f2e4c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:47.289985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f044f27-b758-4d57-9212-d5e06f2e4c14 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:47.291009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f044f27-b758-4d57-9212-d5e06f2e4c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:47.291676 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f044f27-b758-4d57-9212-d5e06f2e4c14 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:47.292514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f044f27-b758-4d57-9212-d5e06f2e4c14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:47.294098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f044f27-b758-4d57-9212-d5e06f2e4c14 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:47.320467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 448/899] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:01:47 2022] POST /identity/v3/services => generated 241 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:01:47.334190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04450c4-e4ce-46bf-9a02-371488edd866 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:47.385745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762fcdd-0d13-4a14-bd64-954857887e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:47.386558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762fcdd-0d13-4a14-bd64-954857887e63 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:47.386959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762fcdd-0d13-4a14-bd64-954857887e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:47.387337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762fcdd-0d13-4a14-bd64-954857887e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:47.471458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762fcdd-0d13-4a14-bd64-954857887e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:47.472896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762fcdd-0d13-4a14-bd64-954857887e63 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:47.534111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:47.584049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 452/900] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:47 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:01:47.590455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f044f27-b758-4d57-9212-d5e06f2e4c14 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:47.698301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f044f27-b758-4d57-9212-d5e06f2e4c14 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:47.734437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:47.735246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:47.735522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:47.735986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:47.752773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:47.754226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:47.770521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b2aee-a11c-43ea-be6b-2acf200134db None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:47.770845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b2aee-a11c-43ea-be6b-2acf200134db None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:47.771074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b2aee-a11c-43ea-be6b-2acf200134db None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:47.772576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 453/901] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:47.776046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:47.776987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:47.777839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:47.777839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:47.777839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:47.777839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 449/902] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:47.782335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112663f-c257-4ca6-8dbe-4823ff758113 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:47.782616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112663f-c257-4ca6-8dbe-4823ff758113 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:47.782892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112663f-c257-4ca6-8dbe-4823ff758113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:47.785417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:47.802251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c112663f-c257-4ca6-8dbe-4823ff758113 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:47.803557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:47.803781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c112663f-c257-4ca6-8dbe-4823ff758113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:47.897146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 454/903] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:47.911221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cdb4b-658a-4fb3-894a-4177fb2b9517 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:47.920049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112663f-c257-4ca6-8dbe-4823ff758113 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:47.950901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:47.952567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:47.953103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:47.953798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:47.955391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:47.955906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:47.958430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:47.974817 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 450/904] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:47.980760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:48.025405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:48.045158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112663f-c257-4ca6-8dbe-4823ff758113 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:48.082553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:48.092589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:48.093223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:48.093494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:48.093766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:48.094369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:48.095706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:48.105749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:48.106705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:48.107596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:48.111786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 455/905] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Aug 9 19:01:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:48.112658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:48.131497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64545e33-f2fd-4e27-a6f0-05dd24bda9f1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:48.180540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:48.182157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:48.182928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:48.183409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:48.193468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:48.195280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:48.209757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:48.210698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:48.211890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 451/906] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:48.216804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:48.226831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33606396-1223-416e-aa94-0e92541d8336 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:48.267054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:48.268188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:48.268697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:48.269169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:48.270021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:48.270653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:48.272350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:48.278395 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 456/907] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:48.283585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:48.301073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:48.326005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8319668-f3f7-490b-91b1-d7eae2a3a28d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:48.358001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:48.364361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:48.365058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:48.365340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:48.365614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:48.367139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:48.367467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:48.369511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:48.382402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 452/908] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:48.393212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:48.401174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95262beb-af58-48ce-8450-93aff55cdea3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:48.437981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:48.438941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:48.439416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:48.439885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:48.440793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:48.441295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:48.442905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:48.448945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 457/909] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:48.454509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:48.517235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee550972-314c-430b-b5a9-c6f389060eee None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:48.548387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:48.555463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:48.556519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:48.556941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:48.557232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:48.557971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] RBAC: Adding query filter params (role.id=8e5e78234e364f77a4017d19ebe64dc0, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=87b647f3da7246b0b530a65c500d5064) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:48.558284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:48.559981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:48.566572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 453/910] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:48 2022] GET /identity/v3/role_assignments?user.id=87b647f3da7246b0b530a65c500d5064&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=8e5e78234e364f77a4017d19ebe64dc0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:48.585153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f5aba-43bb-40b7-9bff-2052561e4698 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:48.627691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a2608-0996-4cf2-993e-6807fc94668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:48.628653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2608-0996-4cf2-993e-6807fc94668f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:48.629110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2608-0996-4cf2-993e-6807fc94668f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:48.629556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2608-0996-4cf2-993e-6807fc94668f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:48.630470 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a2608-0996-4cf2-993e-6807fc94668f None admin] RBAC: Adding query filter params (interface=public, service_id=9138d7bd65454b4e9923fd7d26a973a9, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:48.630962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a2608-0996-4cf2-993e-6807fc94668f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:48.632556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a2608-0996-4cf2-993e-6807fc94668f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:48.642993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 458/911] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Aug 9 19:01:48 2022] GET /identity/v3/endpoints?service_id=9138d7bd65454b4e9923fd7d26a973a9&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:48.648230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:48.718179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:48.768203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b253ea-70dd-4689-906d-d7d2762bf7d8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:48.802947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaf14e-903a-48a6-9bb7-1633e11ab67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:48.803486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaf14e-903a-48a6-9bb7-1633e11ab67c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:48.803767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaf14e-903a-48a6-9bb7-1633e11ab67c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:48.804040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaf14e-903a-48a6-9bb7-1633e11ab67c None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:48.804671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcaf14e-903a-48a6-9bb7-1633e11ab67c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:48.806660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcaf14e-903a-48a6-9bb7-1633e11ab67c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:48.820936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 454/912] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:01:48 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:01:49.218955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a66a4-2b73-49ba-9b39-35dd1312d09b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:49.219354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a66a4-2b73-49ba-9b39-35dd1312d09b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:49.219482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a66a4-2b73-49ba-9b39-35dd1312d09b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:49.221096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 459/913] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:49.226333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc6a3-513d-4904-abb8-4538372b348b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:49.226618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc6a3-513d-4904-abb8-4538372b348b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:49.226893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc6a3-513d-4904-abb8-4538372b348b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:49.246882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f7dcc6a3-513d-4904-abb8-4538372b348b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:49.248052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f7dcc6a3-513d-4904-abb8-4538372b348b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:49.324824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 455/914] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:49.347080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a66a4-2b73-49ba-9b39-35dd1312d09b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:49.467230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a66a4-2b73-49ba-9b39-35dd1312d09b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:49.503421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cab32-c3f0-4401-88cc-a0d9d2063020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:49.503962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cab32-c3f0-4401-88cc-a0d9d2063020 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:49.504233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cab32-c3f0-4401-88cc-a0d9d2063020 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:49.504605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cab32-c3f0-4401-88cc-a0d9d2063020 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:49.505190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cab32-c3f0-4401-88cc-a0d9d2063020 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:49.506601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cab32-c3f0-4401-88cc-a0d9d2063020 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b47cab32-c3f0-4401-88cc-a0d9d2063020 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:49.516174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:49.517171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:49.518038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 460/915] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:49.523380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc6a3-513d-4904-abb8-4538372b348b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:49.528361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c6bd-7b5d-45d2-8113-1ec5c4df59fb None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:49.529249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c6bd-7b5d-45d2-8113-1ec5c4df59fb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:49.529569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c6bd-7b5d-45d2-8113-1ec5c4df59fb None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:49.531141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 461/916] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:49.537090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d55a-aac2-4619-9b9f-db245e02b690 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:49.537364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d55a-aac2-4619-9b9f-db245e02b690 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:49.537628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d55a-aac2-4619-9b9f-db245e02b690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:49.546119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:49.558979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c596d55a-aac2-4619-9b9f-db245e02b690 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:49.560546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c596d55a-aac2-4619-9b9f-db245e02b690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:49.586524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:49.637231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 462/917] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:01:49.647520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc6a3-513d-4904-abb8-4538372b348b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:49.661109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596d55a-aac2-4619-9b9f-db245e02b690 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:49.685196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:49.686127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:49.686481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:49.686977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:49.687688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:49.688101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:49.689814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:49.695226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 456/918] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:49.702128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:49.784913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596d55a-aac2-4619-9b9f-db245e02b690 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:49.824038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:49.824837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:49.825196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:49.825534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:49.826183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:49.827684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:49.830655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:49.839136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6a863-e0af-44b3-958f-feaa68333d89 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:49.840638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:49.841863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:49.842857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:49.845224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 463/919] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:49.850732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:49.854213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:49.884096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:49.885306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:49.885811 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:49.886108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:49.886988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:49.887732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:49.889420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:49.900607 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 457/920] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Aug 9 19:01:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:49.902882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:49.982084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513052-f073-465d-997b-d58e91781e91 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:50.018578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:50.019495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.019931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.020354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.021369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:50.021903 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:50.023465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:50.029837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 464/921] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:50.031624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c446f2-289c-426f-ae56-75cafb6aaf11 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:50.036538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:50.078935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08d51c-6d11-4133-b305-a2a9ef9a6bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:50.080163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08d51c-6d11-4133-b305-a2a9ef9a6bb7 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.080570 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08d51c-6d11-4133-b305-a2a9ef9a6bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.080936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08d51c-6d11-4133-b305-a2a9ef9a6bb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.082081 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08d51c-6d11-4133-b305-a2a9ef9a6bb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:50.083850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08d51c-6d11-4133-b305-a2a9ef9a6bb7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:50.104657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:50.117042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 458/922] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:01:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:01:50.157221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beb84f-780b-4903-8b39-7abf442f1f34 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:50.203255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:50.203875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.204203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.204504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.205492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:50.205802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:50.207176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:50.213743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 465/923] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:50.219824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08d51c-6d11-4133-b305-a2a9ef9a6bb7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:50.264560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:50.341457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08d51c-6d11-4133-b305-a2a9ef9a6bb7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:50.381481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18f994-b5ae-481c-bc82-09e36a5a35bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:50.382100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18f994-b5ae-481c-bc82-09e36a5a35bc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.382375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18f994-b5ae-481c-bc82-09e36a5a35bc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.382643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18f994-b5ae-481c-bc82-09e36a5a35bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.383414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18f994-b5ae-481c-bc82-09e36a5a35bc None admin] RBAC: Adding query filter params (interface=public, service_id=e80f4bcce7db4da482e05bbc218dab22, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:50.383726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18f994-b5ae-481c-bc82-09e36a5a35bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:50.385565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18f994-b5ae-481c-bc82-09e36a5a35bc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:50.393949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 459/924] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Aug 9 19:01:50 2022] GET /identity/v3/endpoints?service_id=e80f4bcce7db4da482e05bbc218dab22&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:50.403272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:50.509938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ff7f1-753c-4fe4-a647-f23d2bf4382b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:50.533842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:50.546603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04669802-71c9-4264-800b-0d126121610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:50.547275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04669802-71c9-4264-800b-0d126121610a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.547554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04669802-71c9-4264-800b-0d126121610a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.547822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04669802-71c9-4264-800b-0d126121610a None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.548503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04669802-71c9-4264-800b-0d126121610a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:50.549884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04669802-71c9-4264-800b-0d126121610a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:50.557210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 466/925] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:01:50 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:01:50.702768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22088e8-a777-470b-b8ca-6b7898a22ffc None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.703322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22088e8-a777-470b-b8ca-6b7898a22ffc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.703322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22088e8-a777-470b-b8ca-6b7898a22ffc None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.704897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 460/926] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:50.713412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227adca9-3b4e-4c59-b07c-468c482b2ef4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.713865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227adca9-3b4e-4c59-b07c-468c482b2ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.714231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227adca9-3b4e-4c59-b07c-468c482b2ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.733488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-227adca9-3b4e-4c59-b07c-468c482b2ef4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:50.734693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-227adca9-3b4e-4c59-b07c-468c482b2ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:50.810537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 467/927] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:50.832864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22088e8-a777-470b-b8ca-6b7898a22ffc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:50.939585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:50.945004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22088e8-a777-470b-b8ca-6b7898a22ffc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:50.973539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae5fe-d8b3-4dfe-aab0-d5a1580e5105 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.974119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae5fe-d8b3-4dfe-aab0-d5a1580e5105 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.974662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae5fe-d8b3-4dfe-aab0-d5a1580e5105 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.976357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 468/928] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:50.982909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d37d2-3fb8-4499-ab9b-f0eec0dd6962 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.983195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d37d2-3fb8-4499-ab9b-f0eec0dd6962 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.983471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d37d2-3fb8-4499-ab9b-f0eec0dd6962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.983591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db4404-703f-4e3f-b360-b85d6207e687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:50.984102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db4404-703f-4e3f-b360-b85d6207e687 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:50.984378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db4404-703f-4e3f-b360-b85d6207e687 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:50.984679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db4404-703f-4e3f-b360-b85d6207e687 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:50.996834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db4404-703f-4e3f-b360-b85d6207e687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f6db4404-703f-4e3f-b360-b85d6207e687 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:51.002368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:51.003311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:51.004160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 461/929] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Aug 9 19:01:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:51.010340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db4404-703f-4e3f-b360-b85d6207e687 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.011137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bf0d37d2-3fb8-4499-ab9b-f0eec0dd6962 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:51.012701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bf0d37d2-3fb8-4499-ab9b-f0eec0dd6962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:51.128497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 469/930] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:51.142187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db4404-703f-4e3f-b360-b85d6207e687 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:51.149148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d37d2-3fb8-4499-ab9b-f0eec0dd6962 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.188615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:51.189613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:51.190118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:51.190609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:51.191478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:51.192000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:51.194112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:51.200963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:51.210438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 462/931] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Aug 9 19:01:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:51.211391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.237280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:51.271291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d37d2-3fb8-4499-ab9b-f0eec0dd6962 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:51.320550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:51.321283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:51.322162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:51.322499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:51.323158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:51.324745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:51.328793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881fa26-b799-4b0a-b76a-e57070f8538f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:51.333835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:51.334794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 19:01:51.335677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:51.340945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 470/932] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Aug 9 19:01:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:51.341961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.387475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:51.387829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:51.388894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:51.388894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:51.406166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:51.409065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:51.435277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:51.436309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:51.437241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:51.437241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:51.437241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:51.439399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 463/933] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:51.445192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.474046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dac591-7162-4d8f-9a89-2f45f390da00 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:51.489178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:51.517361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:51.518024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:51.518236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:51.518491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:51.519180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:51.519490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:51.520905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:51.530843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 471/934] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:51.531680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.543014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:51.577143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fedac7-93aa-40dd-abc6-f6a5fb663f55 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:51.613331 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:51.613938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:51.614215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:51.614488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:51.615124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:51.615429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:51.616922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:51.635984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 464/935] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:51.636910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.675020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcdfa2-7bb0-461d-9d45-97456cd5d757 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:51.726104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:51.727356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:51.728509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:51.729295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:51.730161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:51.730870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:51.732872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:51.740353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 472/936] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:51.747509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.754940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5164ea-83a2-44f4-903c-82a78bd9b753 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:51.795853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:51.798492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:51.804737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:51.805662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:51.806864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:51.807357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:51.816379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:51.818218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:51.831261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:51.832280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:51.833594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:51.835717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 465/937] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:01:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:51.842351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:51.871070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27542d10-1f6c-48e9-be6b-617c1ca73962 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:51.914877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ba8f2-5b30-46d3-bf82-dddefcb92d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:51.916663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba8f2-5b30-46d3-bf82-dddefcb92d55 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:51.916961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba8f2-5b30-46d3-bf82-dddefcb92d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:51.917237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba8f2-5b30-46d3-bf82-dddefcb92d55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:51.917905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ba8f2-5b30-46d3-bf82-dddefcb92d55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:51.919548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ba8f2-5b30-46d3-bf82-dddefcb92d55 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:51.955161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 473/938] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:01:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:51.961113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c1fa-613f-41cf-879f-727adc54effb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:52.011476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:52.012055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.012388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.012701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.013441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:52.013752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:52.015278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:52.037101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 466/939] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Aug 9 19:01:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:52.045642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ba8f2-5b30-46d3-bf82-dddefcb92d55 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:52.152784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:52.161448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ba8f2-5b30-46d3-bf82-dddefcb92d55 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:52.200270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63eca33-7747-4c6c-a001-23925a73973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:52.200951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63eca33-7747-4c6c-a001-23925a73973e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.201227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63eca33-7747-4c6c-a001-23925a73973e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.201494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63eca33-7747-4c6c-a001-23925a73973e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.213461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63eca33-7747-4c6c-a001-23925a73973e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:52.214922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63eca33-7747-4c6c-a001-23925a73973e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f63eca33-7747-4c6c-a001-23925a73973e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:52.230635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:52.231634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:52.232588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:52.232588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:52.232588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.232588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 474/940] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:52.237772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:52.291787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:52.297349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bd143-b973-4ce9-af02-49feda9ad4ec None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.297641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bd143-b973-4ce9-af02-49feda9ad4ec None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.297899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bd143-b973-4ce9-af02-49feda9ad4ec None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.299501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 475/941] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:52.308462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b1871-fba3-4d9f-b5a6-fc105d04a463 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.308462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b1871-fba3-4d9f-b5a6-fc105d04a463 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.308462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b1871-fba3-4d9f-b5a6-fc105d04a463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.329065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-784b1871-fba3-4d9f-b5a6-fc105d04a463 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:52.330251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-784b1871-fba3-4d9f-b5a6-fc105d04a463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:52.356783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af481edf-9b3b-40df-88f6-6cb06d1e2082 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:52.404002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:52.404726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.405091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.405430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.406145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:52.406514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:52.407936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:52.425776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 467/943] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:52.426779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:52.428652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 476/943] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:01:52.456476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b1871-fba3-4d9f-b5a6-fc105d04a463 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:52.550479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:52.555064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27c19e-e615-4a6d-bf81-03bd1245ad57 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:52.565044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:52.579847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b1871-fba3-4d9f-b5a6-fc105d04a463 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:52.596241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:52.596959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.597272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.597510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.604842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:52.606390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:52.618085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:52.619236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:52.620229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:52.620229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:52.620229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:52.620229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:52.620229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:52.620229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:52.620229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:52.620229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:52.624535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 468/944] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:52.625597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:52.626237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:52.627046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.627466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.627881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.628890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:52.631402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:52.634739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING keystone.server.flask.common [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 19:01:52.659327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.sql.core [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 19:01:52.659327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 19:01:52.659327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [parameters: {'id': 'e5c7da33c8b04ed6bec877b479880733', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 19:01:52.659327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80339) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e5c7da33c8b04ed6bec877b479880733.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e5c7da33c8b04ed6bec877b479880733. Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application conn.query(q) Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 19:01:52.681641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result.read() Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 09 19:01:52.682587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application next(self.gen) Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application yield resource Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application next(self.gen) Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise self.value Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application session.commit() Aug 09 19:01:52.683538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.session.flush() Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application compat.raise_( Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 19:01:52.684704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 19:01:52.685608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application conn.query(q) Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result.read() Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 19:01:52.686362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [parameters: {'id': 'e5c7da33c8b04ed6bec877b479880733', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:52.687210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:52.689887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e5c7da33c8b04ed6bec877b479880733. Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 477/945] 10.222.0.70 () {60 vars in 1140 bytes} [Tue Aug 9 19:01:52 2022] POST /identity/v3/projects => generated 262 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:52.692043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:52.754843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017bfb-a500-490b-9e75-24353a343ed6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:52.792514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8563b59-a90d-4452-9eba-5b80d347ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:52.793202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563b59-a90d-4452-9eba-5b80d347ea56 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.793391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563b59-a90d-4452-9eba-5b80d347ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.793666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563b59-a90d-4452-9eba-5b80d347ea56 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.794359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8563b59-a90d-4452-9eba-5b80d347ea56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:52.794663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8563b59-a90d-4452-9eba-5b80d347ea56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:52.796086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8563b59-a90d-4452-9eba-5b80d347ea56 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:52.813844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acfefd-3e94-4fd5-bdbb-7499e28dc033 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:52.814009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 469/946] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:52.814009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92981010-3d00-416e-bcd5-69a5a51a69ce None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.814009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92981010-3d00-416e-bcd5-69a5a51a69ce None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.814009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92981010-3d00-416e-bcd5-69a5a51a69ce None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.814009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 470/947] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:52.817079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92981010-3d00-416e-bcd5-69a5a51a69ce None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:52.846735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:52.847430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:52.854907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc71f69-42bd-45e5-a9c7-da29cebd07ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:52.855530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71f69-42bd-45e5-a9c7-da29cebd07ac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.855811 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71f69-42bd-45e5-a9c7-da29cebd07ac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.856600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71f69-42bd-45e5-a9c7-da29cebd07ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.864951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc71f69-42bd-45e5-a9c7-da29cebd07ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:52.866604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc71f69-42bd-45e5-a9c7-da29cebd07ac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fcc71f69-42bd-45e5-a9c7-da29cebd07ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:52.878572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:52.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:52.880348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 478/948] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:52.881455 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707ce67-0d45-4e12-b4fe-a8e7e96dd21c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.881455 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707ce67-0d45-4e12-b4fe-a8e7e96dd21c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.881455 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707ce67-0d45-4e12-b4fe-a8e7e96dd21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.899963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8707ce67-0d45-4e12-b4fe-a8e7e96dd21c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:52.901775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8707ce67-0d45-4e12-b4fe-a8e7e96dd21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:52.935660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92981010-3d00-416e-bcd5-69a5a51a69ce None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:52.972690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 479/949] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:52.973637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e70e8-8729-442a-9558-dddc865ac0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:52.974247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e70e8-8729-442a-9558-dddc865ac0d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:52.974523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e70e8-8729-442a-9558-dddc865ac0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:52.974794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e70e8-8729-442a-9558-dddc865ac0d1 None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/87b647f3da7246b0b530a65c500d5064/roles/8e5e78234e364f77a4017d19ebe64dc0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:52.976944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e70e8-8729-442a-9558-dddc865ac0d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=87b647f3da7246b0b530a65c500d5064, role_id=8e5e78234e364f77a4017d19ebe64dc0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:52.977988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707ce67-0d45-4e12-b4fe-a8e7e96dd21c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:52.978552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e70e8-8729-442a-9558-dddc865ac0d1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:52.997287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 471/950] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:01:52 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/87b647f3da7246b0b530a65c500d5064/roles/8e5e78234e364f77a4017d19ebe64dc0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:01:52.998098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e70e8-8729-442a-9558-dddc865ac0d1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:53.099804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:53.103818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707ce67-0d45-4e12-b4fe-a8e7e96dd21c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:53.117697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e70e8-8729-442a-9558-dddc865ac0d1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:53.135271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:53.153850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:53.154738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:53.155120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:53.155492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:53.156372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:53.156819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:53.158616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:53.161799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:53.162636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:53.163083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:53.163587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:53.164579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:53.166269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:53.176392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:53.177449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:53.178280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:53.181551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 472/951] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:53.183995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 480/952] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:53.188938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:53.311254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586b2a9-3ff5-46c4-b01c-cdb0e31717ed None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:53.348994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:53.349596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:53.349940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:53.350158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:53.350813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:53.351096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:53.352474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:53.358160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 473/953] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:53.363716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:53.482794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d52dc-fe4d-4257-9648-21d71b88fac2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:53.521217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677412c0-9558-43b3-804e-10025fb15147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:53.521830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677412c0-9558-43b3-804e-10025fb15147 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:53.522107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677412c0-9558-43b3-804e-10025fb15147 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:53.522377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677412c0-9558-43b3-804e-10025fb15147 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:53.523006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677412c0-9558-43b3-804e-10025fb15147 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:53.523304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677412c0-9558-43b3-804e-10025fb15147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:53.524786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677412c0-9558-43b3-804e-10025fb15147 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:53.530832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 481/954] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:53.537511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:53.558447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:53.650436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c00aa-2334-4127-bf0d-f2ef92ee3a7a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:53.685783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fbf38-646b-49b9-84bc-c28baaf8446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:53.686787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fbf38-646b-49b9-84bc-c28baaf8446c None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:53.687270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fbf38-646b-49b9-84bc-c28baaf8446c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:53.687734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fbf38-646b-49b9-84bc-c28baaf8446c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:53.688610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fbf38-646b-49b9-84bc-c28baaf8446c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:53.690153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fbf38-646b-49b9-84bc-c28baaf8446c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:53.713506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 474/955] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:01:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:54.232531 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4af0f8-e35e-47a3-8b68-73d55cd76030 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:54.233513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4af0f8-e35e-47a3-8b68-73d55cd76030 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:54.233995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4af0f8-e35e-47a3-8b68-73d55cd76030 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:54.235796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 482/956] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:54.241074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58e3cf-cd8b-4c21-9adf-9680f7fc5600 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:54.241874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58e3cf-cd8b-4c21-9adf-9680f7fc5600 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:54.242354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58e3cf-cd8b-4c21-9adf-9680f7fc5600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:54.261721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bf58e3cf-cd8b-4c21-9adf-9680f7fc5600 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:54.263264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bf58e3cf-cd8b-4c21-9adf-9680f7fc5600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:54.330865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:54.342647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 475/957] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:54.364544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4af0f8-e35e-47a3-8b68-73d55cd76030 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:54.423967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:54.487327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4af0f8-e35e-47a3-8b68-73d55cd76030 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:54.527590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:54.528234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:54.528544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:54.528885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:54.529646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:54.531021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:54.540501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:54.542157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:54.544374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 483/958] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:54.547704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58e3cf-cd8b-4c21-9adf-9680f7fc5600 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:54.650874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58e3cf-cd8b-4c21-9adf-9680f7fc5600 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:54.686291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6fd0c-e6d6-4bea-a267-238e1cdf0a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:54.686964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6fd0c-e6d6-4bea-a267-238e1cdf0a8c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:54.687246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6fd0c-e6d6-4bea-a267-238e1cdf0a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:54.687513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6fd0c-e6d6-4bea-a267-238e1cdf0a8c None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:54.688135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6fd0c-e6d6-4bea-a267-238e1cdf0a8c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:54.688449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6fd0c-e6d6-4bea-a267-238e1cdf0a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:54.689804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6fd0c-e6d6-4bea-a267-238e1cdf0a8c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:54.694959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 476/959] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:54.701203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:54.809134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91029b-717a-4d7e-9393-6cf7969083e1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:54.825350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:54.850500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add559d0-3d4b-45aa-bd62-18d5c4c9e8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:54.851108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add559d0-3d4b-45aa-bd62-18d5c4c9e8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:54.851388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add559d0-3d4b-45aa-bd62-18d5c4c9e8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:54.851655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add559d0-3d4b-45aa-bd62-18d5c4c9e8a0 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:54.852283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add559d0-3d4b-45aa-bd62-18d5c4c9e8a0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:54.852658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add559d0-3d4b-45aa-bd62-18d5c4c9e8a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:54.854184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add559d0-3d4b-45aa-bd62-18d5c4c9e8a0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:54.859433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 484/960] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:55.268020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71019-3af7-420c-bcd9-5ab40254b69c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.268748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71019-3af7-420c-bcd9-5ab40254b69c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.268748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71019-3af7-420c-bcd9-5ab40254b69c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.270029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 477/961] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:55.275443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3d672-aa7c-4543-adff-da7832a8299c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.275957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3d672-aa7c-4543-adff-da7832a8299c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.276124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3d672-aa7c-4543-adff-da7832a8299c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.294610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b1c3d672-aa7c-4543-adff-da7832a8299c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:55.295795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b1c3d672-aa7c-4543-adff-da7832a8299c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:55.367564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 485/962] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:55.392695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a71019-3af7-420c-bcd9-5ab40254b69c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:55.464033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:55.505596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e886-d4f6-404d-b93a-5e456b6d886f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.505920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e886-d4f6-404d-b93a-5e456b6d886f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.506209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e886-d4f6-404d-b93a-5e456b6d886f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.507955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 486/963] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:55.508188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a71019-3af7-420c-bcd9-5ab40254b69c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:55.515527 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551f049-218f-43e6-b462-228ddf12637a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.516134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551f049-218f-43e6-b462-228ddf12637a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.516638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551f049-218f-43e6-b462-228ddf12637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.539738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1551f049-218f-43e6-b462-228ddf12637a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:55.542496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1551f049-218f-43e6-b462-228ddf12637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:55.552069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ae5d8-689e-43d6-9369-0eba63a24b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:55.552717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae5d8-689e-43d6-9369-0eba63a24b74 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.552994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae5d8-689e-43d6-9369-0eba63a24b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.553250 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae5d8-689e-43d6-9369-0eba63a24b74 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.568901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ae5d8-689e-43d6-9369-0eba63a24b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2b5ae5d8-689e-43d6-9369-0eba63a24b74 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:01:55.575465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:55.576408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:55.577282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 478/964] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Aug 9 19:01:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:55.582628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ae5d8-689e-43d6-9369-0eba63a24b74 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:55.644461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 487/965] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:55.661969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551f049-218f-43e6-b462-228ddf12637a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:55.699779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:55.717245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ae5d8-689e-43d6-9369-0eba63a24b74 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:55.742158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:55.758848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:55.759454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.759799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.760187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.760894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:55.761196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:55.762547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:55.771346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 479/966] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Aug 9 19:01:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:55.775168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551f049-218f-43e6-b462-228ddf12637a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:55.783686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:55.817754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3ee3f-11d7-42b5-bd19-2a38ecec9d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:55.819318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3ee3f-11d7-42b5-bd19-2a38ecec9d6b None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.819802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3ee3f-11d7-42b5-bd19-2a38ecec9d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.820426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3ee3f-11d7-42b5-bd19-2a38ecec9d6b None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.822155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3ee3f-11d7-42b5-bd19-2a38ecec9d6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:55.824773 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3ee3f-11d7-42b5-bd19-2a38ecec9d6b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:55.842641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.sql.core [None req-b3e3ee3f-11d7-42b5-bd19-2a38ecec9d6b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 19:01:55.842641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 19:01:55.842641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [parameters: {'user_id': '39b4bcceea65456c9c2883b576116868', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 19:01:55.842641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80339) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b3e3ee3f-11d7-42b5-bd19-2a38ecec9d6b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application conn.query(q) Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result.read() Aug 09 19:01:55.849952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 19:01:55.850825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application next(self.gen) Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application yield resource Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application next(self.gen) Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise self.value Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application session.commit() Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 19:01:55.851765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.session.flush() Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application compat.raise_( Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 19:01:55.853480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 19:01:55.854743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application conn.query(q) Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application result.read() Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 19:01:55.855638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [parameters: {'user_id': '39b4bcceea65456c9c2883b576116868', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:55.858734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 09 19:01:55.860767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 488/967] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Aug 9 19:01:55 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bebc3c-e713-4709-96b5-e15fee0a7534 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bebc3c-e713-4709-96b5-e15fee0a7534 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.863628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bebc3c-e713-4709-96b5-e15fee0a7534 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.865195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 489/968] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:55.865195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bebc3c-e713-4709-96b5-e15fee0a7534 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:55.911850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fd3fd-02d8-4321-95d8-41065c4e578b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:55.951946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1691e5-f3a5-4b24-b22d-ed50b0f32dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:55.952606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1691e5-f3a5-4b24-b22d-ed50b0f32dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.952881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1691e5-f3a5-4b24-b22d-ed50b0f32dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.953150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1691e5-f3a5-4b24-b22d-ed50b0f32dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:55.964969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1691e5-f3a5-4b24-b22d-ed50b0f32dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:55.966385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1691e5-f3a5-4b24-b22d-ed50b0f32dc1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:55.966774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:55.975636 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bebc3c-e713-4709-96b5-e15fee0a7534 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-af1691e5-f3a5-4b24-b22d-ed50b0f32dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:55.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:55.983593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:55.984521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:55.984521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:55.984521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:55.988429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 480/969] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:55.989590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686de97-fa6b-492e-9e1e-e8ded39ae695 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:55.989869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686de97-fa6b-492e-9e1e-e8ded39ae695 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:55.990141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686de97-fa6b-492e-9e1e-e8ded39ae695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.013391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.014026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.014305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.014571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.015719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6686de97-fa6b-492e-9e1e-e8ded39ae695 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:56.017983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6686de97-fa6b-492e-9e1e-e8ded39ae695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:56.022165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.023733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:56.037398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:56.038426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 490/970] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:01:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:56.039336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.059839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:56.109106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 481/971] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:56.109106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686de97-fa6b-492e-9e1e-e8ded39ae695 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.161149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d7a56-5d8f-4fe3-942e-b934048555d9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.195680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.196946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.197343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.197738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.198480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:56.198850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.200294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.218198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 491/972] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:56.218198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.225110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686de97-fa6b-492e-9e1e-e8ded39ae695 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.236907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:56.260585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.261274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.261625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.261967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.262735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:56.263114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.264653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.304476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 482/973] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:56.304476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.337921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:56.345117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54abd62-0684-4776-b60e-01e6652a7ec9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.393196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.394127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.394510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.394883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.395646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.397189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:56.408717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:56.410030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:56.411069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:56.413105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 492/974] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:56.419587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.441915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69816b-b826-473f-a0a7-1d79ab0cb25c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.492755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.494005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.494643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.495045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.504664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.506237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:01:56.520803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:56.521986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 19:01:56.523152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:56.526572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 483/975] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:56.534768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:56.536293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.564282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c49b7a-51c2-4f4f-a8ea-ca803fb38f4a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.613822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.614924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.615353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.615922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.617090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:56.617392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.619357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.625953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 493/976] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:56.633570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.644629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:56.667583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e09506-fe9e-443c-b178-6a951f620b53 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.708317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.709387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.709896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.710391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.711739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:56.712796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.715045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.738186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 484/977] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:56.739268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.752034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130cc67-24c4-4dcc-80a4-0a20af251fff None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.788758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.789499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.789779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.790664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.791636 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:56.791945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.793544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.804941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 494/978] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:56.806042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:56.818242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:56.847618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bea738-b9d2-406a-9ef6-be75abe47348 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.885574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babfcd9-0c08-40fd-a68e-d878e046b617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.886181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babfcd9-0c08-40fd-a68e-d878e046b617 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.886458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babfcd9-0c08-40fd-a68e-d878e046b617 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.886729 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babfcd9-0c08-40fd-a68e-d878e046b617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.899280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:56.916515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30dda3-798f-44a2-91f8-bd01f33f4af3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:56.956973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc30c5b-7e36-4a0e-a728-286a576802c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:56.957866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc30c5b-7e36-4a0e-a728-286a576802c4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.958139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc30c5b-7e36-4a0e-a728-286a576802c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.958410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc30c5b-7e36-4a0e-a728-286a576802c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.959110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc30c5b-7e36-4a0e-a728-286a576802c4 None admin] RBAC: Adding query filter params (interface=admin, service_id=e80f4bcce7db4da482e05bbc218dab22, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:56.959413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc30c5b-7e36-4a0e-a728-286a576802c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.960790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc30c5b-7e36-4a0e-a728-286a576802c4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.965699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babfcd9-0c08-40fd-a68e-d878e046b617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:56.970592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 495/979] 10.222.0.70 () {56 vars in 1249 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/endpoints?service_id=e80f4bcce7db4da482e05bbc218dab22&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:56.970739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babfcd9-0c08-40fd-a68e-d878e046b617 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:56.970816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73a46b-0e64-4e15-aef9-1b2bd9f4c5cd None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:56.970816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73a46b-0e64-4e15-aef9-1b2bd9f4c5cd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:56.970816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73a46b-0e64-4e15-aef9-1b2bd9f4c5cd None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:56.972564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 496/980] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:56.973397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73a46b-0e64-4e15-aef9-1b2bd9f4c5cd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:57.050369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:57.071050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 485/981] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 19:01:57.072424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca464c-4921-44fa-953d-3bc8e4e88fed None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:57.072792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca464c-4921-44fa-953d-3bc8e4e88fed None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:57.073065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca464c-4921-44fa-953d-3bc8e4e88fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:57.090766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73a46b-0e64-4e15-aef9-1b2bd9f4c5cd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:57.091410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-28ca464c-4921-44fa-953d-3bc8e4e88fed None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:57.092624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-28ca464c-4921-44fa-953d-3bc8e4e88fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:57.126859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069dc91-48c7-4e96-bc4f-5b3715062dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:57.127434 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:57.127552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069dc91-48c7-4e96-bc4f-5b3715062dad None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:57.127783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069dc91-48c7-4e96-bc4f-5b3715062dad None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:57.128053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069dc91-48c7-4e96-bc4f-5b3715062dad None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:57.128709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069dc91-48c7-4e96-bc4f-5b3715062dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:57.130272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069dc91-48c7-4e96-bc4f-5b3715062dad None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:57.136224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 497/982] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:01:56 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:01:57.137042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069dc91-48c7-4e96-bc4f-5b3715062dad None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:57.173989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 486/983] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:57.197681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca464c-4921-44fa-953d-3bc8e4e88fed None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:57.259799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069dc91-48c7-4e96-bc4f-5b3715062dad None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:57.300863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:57.301491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:57.301764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:57.302030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:57.306870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:57.308320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca464c-4921-44fa-953d-3bc8e4e88fed None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:57.317307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:57.319051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:01:57.335017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:57.336217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:01:57.337682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:01:57.337682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:01:57.337682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:57.337682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 498/984] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:57.342045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:57.348387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc883f-bf6a-422d-8207-8421368af3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:57.349048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc883f-bf6a-422d-8207-8421368af3b0 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:57.349383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc883f-bf6a-422d-8207-8421368af3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:57.349662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc883f-bf6a-422d-8207-8421368af3b0 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:57.350315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc883f-bf6a-422d-8207-8421368af3b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:57.351680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc883f-bf6a-422d-8207-8421368af3b0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:57.369065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 487/985] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:01:57 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:01:57.466319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ad50-6406-4984-9e21-ee3866d9ce12 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:57.514881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:57.515496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:57.515774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:57.516049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:57.516721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:57.517026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:57.518406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:57.532614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 499/986] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:57.538219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc883f-bf6a-422d-8207-8421368af3b0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:57.579679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:57.658785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc883f-bf6a-422d-8207-8421368af3b0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:57.692806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:57.693540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:57.693820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:57.694091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:57.700845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:57.702294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:01:57.714366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:57.715278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:57.716183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 488/987] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:57.721467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:57.770419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:57.827021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f04f-55ef-46dd-bd99-e92d761dca55 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:57.862441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7231e3-e319-46b3-83a0-3f511d85f898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:57.863049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7231e3-e319-46b3-83a0-3f511d85f898 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:57.863325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7231e3-e319-46b3-83a0-3f511d85f898 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:57.863594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7231e3-e319-46b3-83a0-3f511d85f898 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:57.864282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7231e3-e319-46b3-83a0-3f511d85f898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:57.864613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7231e3-e319-46b3-83a0-3f511d85f898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:57.866086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7231e3-e319-46b3-83a0-3f511d85f898 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:57.882868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 500/988] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:57.884912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:57.978647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:57.989506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5cf0e-97f6-4db2-b9a5-25f7589a8cb4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:58.023103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4795b-9b98-4f43-8115-54d5c0925734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:58.023707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4795b-9b98-4f43-8115-54d5c0925734 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:58.023984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4795b-9b98-4f43-8115-54d5c0925734 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:58.024254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4795b-9b98-4f43-8115-54d5c0925734 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:58.025721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4795b-9b98-4f43-8115-54d5c0925734 None admin] RBAC: Adding query filter params (role.id=8e5e78234e364f77a4017d19ebe64dc0, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=87b647f3da7246b0b530a65c500d5064) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:58.026127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4795b-9b98-4f43-8115-54d5c0925734 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:58.027610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4795b-9b98-4f43-8115-54d5c0925734 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:58.039314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 489/989] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:01:57 2022] GET /identity/v3/role_assignments?user.id=87b647f3da7246b0b530a65c500d5064&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=8e5e78234e364f77a4017d19ebe64dc0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:59.164320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11fa5c-0e99-4b2d-915a-130b5f833a60 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.164882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11fa5c-0e99-4b2d-915a-130b5f833a60 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.164882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11fa5c-0e99-4b2d-915a-130b5f833a60 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.167342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 501/990] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:59.173117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccea03-d6ec-48a3-a172-8ff20f5bf94f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.173615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccea03-d6ec-48a3-a172-8ff20f5bf94f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.173688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccea03-d6ec-48a3-a172-8ff20f5bf94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.192535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d9ccea03-d6ec-48a3-a172-8ff20f5bf94f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:59.193706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d9ccea03-d6ec-48a3-a172-8ff20f5bf94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:59.264411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 490/991] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:01:59.287403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11fa5c-0e99-4b2d-915a-130b5f833a60 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:59.396047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11fa5c-0e99-4b2d-915a-130b5f833a60 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:59.398640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:59.432859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:59.433409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.433687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.433958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.434794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:59.436151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:59.445009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:59.445968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:01:59.446971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 502/992] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Aug 9 19:01:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:01:59.452178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15fea-016c-4ea3-af41-c339dd947735 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.452334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:59.452943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15fea-016c-4ea3-af41-c339dd947735 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.452943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15fea-016c-4ea3-af41-c339dd947735 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.454201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 491/993] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:01:59.458823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551980b-3cad-414d-8885-32024612ad1f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.459109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551980b-3cad-414d-8885-32024612ad1f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.459388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551980b-3cad-414d-8885-32024612ad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.479404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4551980b-3cad-414d-8885-32024612ad1f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:01:59.480625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4551980b-3cad-414d-8885-32024612ad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:01:59.560168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d462f-b84b-4c32-8b7f-30eb5c5919ba None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:59.564524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 492/994] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:01:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:01:59.580569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551980b-3cad-414d-8885-32024612ad1f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:59.597140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:59.598046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:59.598836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.598947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.599297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.599829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:59.600116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:59.601513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:59.606890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 503/995] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:59.612396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:59.645536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:59.693435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551980b-3cad-414d-8885-32024612ad1f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:59.725202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f0b8-72e5-4c76-9a0f-dfd04cbb1cb5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:59.736633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:59.737573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.737853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.738126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.738710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:59.740066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:01:59.749396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:01:59.750487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:01:59.751315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:01:59.755540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 493/996] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:01:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:01:59.756379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:59.766071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:59.766776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.767050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.767321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.767949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:59.768250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:59.769646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:59.780204 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 504/997] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Aug 9 19:01:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:01:59.781081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:59.847431 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:59.871937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1395c7c-7b8e-447c-9288-29d86c3ea1bf None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:59.881465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:01:59.888711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ce34c-b423-4a8b-a8f0-a78a8d7c67fb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:01:59.907296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:59.907908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.908185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.910284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.910284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:01:59.910284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:59.910808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:59.916890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 494/998] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:01:59.922323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:01:59.935972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb3733-1468-4b9c-9d4c-e54f1427c2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:01:59.936624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3733-1468-4b9c-9d4c-e54f1427c2de None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:01:59.936892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3733-1468-4b9c-9d4c-e54f1427c2de None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:01:59.937162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3733-1468-4b9c-9d4c-e54f1427c2de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:01:59.937815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb3733-1468-4b9c-9d4c-e54f1427c2de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:01:59.939180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb3733-1468-4b9c-9d4c-e54f1427c2de None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:01:59.956660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 505/999] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:01:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:02:00.042624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77e391-c4c4-4967-9ca7-dc92cd3fb909 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:00.091583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:00.092577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:00.093026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:00.093384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e9c84-8559-4223-9b91-6c68ec43f025 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:00.093517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:00.093674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e9c84-8559-4223-9b91-6c68ec43f025 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:00.093935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e9c84-8559-4223-9b91-6c68ec43f025 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:00.094110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:00.094369 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:00.095468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 506/1000] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:02:00.095810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:00.100690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffcd79-505b-4147-9dac-d69afd9901b4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:00.100972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffcd79-505b-4147-9dac-d69afd9901b4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:00.101237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffcd79-505b-4147-9dac-d69afd9901b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:00.107179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 495/1001] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:01:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:00.108727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:00.110889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:00.121811 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d6ffcd79-505b-4147-9dac-d69afd9901b4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:00.123399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d6ffcd79-505b-4147-9dac-d69afd9901b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:00.202026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 507/1002] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:00.220705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a04c3-290f-4a18-8296-27fa0730488b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:00.224595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffcd79-505b-4147-9dac-d69afd9901b4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:00.256196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:00.256857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:00.257099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:00.257371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:00.258068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] RBAC: Adding query filter params (interface=public, service_id=a4e72653ed634c9b9315ceb593522db0, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:00.258389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:00.260131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:00.265688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 496/1003] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Aug 9 19:02:00 2022] GET /identity/v3/endpoints?service_id=a4e72653ed634c9b9315ceb593522db0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:00.271066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:00.322643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:00.339386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffcd79-505b-4147-9dac-d69afd9901b4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:00.346934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:00.379440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:00.380377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:00.380675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:00.380944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:00.381523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:00.382889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:00.392521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeae1c-a0df-49f1-9f3b-ce1a97de0cfb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:00.395247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:00.396262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:00.397161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 508/1004] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Aug 9 19:02:00 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:00.403019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:00.436113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7039d4d-25ad-4054-a664-20d9612db5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:00.437332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7039d4d-25ad-4054-a664-20d9612db5fd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:00.437768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7039d4d-25ad-4054-a664-20d9612db5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:00.438190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7039d4d-25ad-4054-a664-20d9612db5fd None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:00.438938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7039d4d-25ad-4054-a664-20d9612db5fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:00.440542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7039d4d-25ad-4054-a664-20d9612db5fd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:00.448485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 497/1005] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:02:00 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:02:00.522270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68418e3-61ee-4e96-b540-24972a047da6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:00.560546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74183b6-cc90-432b-b72a-fdd31e6045d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:00.561501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74183b6-cc90-432b-b72a-fdd31e6045d9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:00.561964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74183b6-cc90-432b-b72a-fdd31e6045d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:00.562431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74183b6-cc90-432b-b72a-fdd31e6045d9 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:00.563227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74183b6-cc90-432b-b72a-fdd31e6045d9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:00.563687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74183b6-cc90-432b-b72a-fdd31e6045d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:00.565267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74183b6-cc90-432b-b72a-fdd31e6045d9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:00.570933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 509/1006] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:00.577692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7039d4d-25ad-4054-a664-20d9612db5fd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:00.685090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7039d4d-25ad-4054-a664-20d9612db5fd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:00.715266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:00.721794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16b874-a833-4ecc-b57f-cb2ad3985f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:00.722406 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b874-a833-4ecc-b57f-cb2ad3985f63 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:00.722681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b874-a833-4ecc-b57f-cb2ad3985f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:00.722950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b874-a833-4ecc-b57f-cb2ad3985f63 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:00.723948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16b874-a833-4ecc-b57f-cb2ad3985f63 None admin] RBAC: Adding query filter params (type=image) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:00.724252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16b874-a833-4ecc-b57f-cb2ad3985f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:00.725633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16b874-a833-4ecc-b57f-cb2ad3985f63 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:00.730662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 498/1007] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:01.988783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49889-04f0-4597-aa30-57f2307b51e0 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:01.989304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49889-04f0-4597-aa30-57f2307b51e0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:01.989304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49889-04f0-4597-aa30-57f2307b51e0 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:01.990769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 510/1008] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:01.995758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eab29-202e-4984-baa1-68b91ad91742 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:01.996148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eab29-202e-4984-baa1-68b91ad91742 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:01.996358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eab29-202e-4984-baa1-68b91ad91742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.013998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7a2eab29-202e-4984-baa1-68b91ad91742 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:02.015161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7a2eab29-202e-4984-baa1-68b91ad91742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:02.084197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 499/1009] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:02.107598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a49889-04f0-4597-aa30-57f2307b51e0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:02.124851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:02.214238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a49889-04f0-4597-aa30-57f2307b51e0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:02.247653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a14a8-7a85-4029-beb6-5276e043995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:02.248405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a14a8-7a85-4029-beb6-5276e043995a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:02.248580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a14a8-7a85-4029-beb6-5276e043995a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:02.248746 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a14a8-7a85-4029-beb6-5276e043995a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.249837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a14a8-7a85-4029-beb6-5276e043995a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:02.251251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a14a8-7a85-4029-beb6-5276e043995a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:02.252982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 511/1010] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:02.258495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eab29-202e-4984-baa1-68b91ad91742 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:02.358804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eab29-202e-4984-baa1-68b91ad91742 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:02.385764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:02.399275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c55908-e8ea-4330-aebc-0f7bb6424003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:02.399880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55908-e8ea-4330-aebc-0f7bb6424003 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:02.400154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55908-e8ea-4330-aebc-0f7bb6424003 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:02.400422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55908-e8ea-4330-aebc-0f7bb6424003 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.401098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c55908-e8ea-4330-aebc-0f7bb6424003 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:02.402507 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c55908-e8ea-4330-aebc-0f7bb6424003 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:02.431671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 500/1011] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:02:02 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:02:02.496915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1561b-c0c1-4fa2-9d09-ea85be87fdf8 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:02.497221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1561b-c0c1-4fa2-9d09-ea85be87fdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:02.497648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1561b-c0c1-4fa2-9d09-ea85be87fdf8 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.499419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 512/1012] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:02.505322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4bde9-0e98-4367-9461-235998d1c508 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:02.505873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4bde9-0e98-4367-9461-235998d1c508 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:02.506338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4bde9-0e98-4367-9461-235998d1c508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.526098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6fa4bde9-0e98-4367-9461-235998d1c508 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:02.527617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6fa4bde9-0e98-4367-9461-235998d1c508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:02.608017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 501/1013] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:02.631392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1561b-c0c1-4fa2-9d09-ea85be87fdf8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:02.721003 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:02.741511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1561b-c0c1-4fa2-9d09-ea85be87fdf8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:02.747107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55f347-5b77-416e-9331-c8ff73737cbe None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:02.747384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55f347-5b77-416e-9331-c8ff73737cbe None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:02.747659 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55f347-5b77-416e-9331-c8ff73737cbe None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.749154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 502/1014] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:02:02.754065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7d9b-caa4-41ee-a038-a7171b3f9c5f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:02.754291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7d9b-caa4-41ee-a038-a7171b3f9c5f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:02.754566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7d9b-caa4-41ee-a038-a7171b3f9c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.772329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d53e7d9b-caa4-41ee-a038-a7171b3f9c5f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:02.773530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d53e7d9b-caa4-41ee-a038-a7171b3f9c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:02.778895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:02.779750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:02.780143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:02.780423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] PATH_INFO: `/v3/services/compute` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.781064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:02.782436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:02.791998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:02.793051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:02.793925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 513/1015] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:02:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:02.799126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:02.851259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 503/1016] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:02.874216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e7d9b-caa4-41ee-a038-a7171b3f9c5f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:02.905577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf36bd6-b851-4b83-85d4-6cc847b075ff None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:02.909527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:02.948485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:02.948485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:02.948485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:02.948485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:02.949093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:02.949301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:02.950668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:02.955796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 514/1017] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:02:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:02.961622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:02.977261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:03.000175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e7d9b-caa4-41ee-a038-a7171b3f9c5f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:03.038642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd85a27-a5c5-4ed7-b0d8-32885e5f7348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:03.039513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd85a27-a5c5-4ed7-b0d8-32885e5f7348 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:03.040163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd85a27-a5c5-4ed7-b0d8-32885e5f7348 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:03.040607 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd85a27-a5c5-4ed7-b0d8-32885e5f7348 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:03.041681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd85a27-a5c5-4ed7-b0d8-32885e5f7348 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:03.044185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd85a27-a5c5-4ed7-b0d8-32885e5f7348 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:03.061424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 504/1018] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:02:02 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:02:03.091359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ed37-fdd9-4528-af43-e9b96a121314 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:03.138060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c6f28-dde4-4b3c-af49-44bf3c29fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:03.138784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c6f28-dde4-4b3c-af49-44bf3c29fe81 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:03.139062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c6f28-dde4-4b3c-af49-44bf3c29fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:03.139332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c6f28-dde4-4b3c-af49-44bf3c29fe81 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:03.140064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c6f28-dde4-4b3c-af49-44bf3c29fe81 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:03.140366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c6f28-dde4-4b3c-af49-44bf3c29fe81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:03.141802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c6f28-dde4-4b3c-af49-44bf3c29fe81 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:03.152565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 515/1019] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Aug 9 19:02:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:03.153854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd85a27-a5c5-4ed7-b0d8-32885e5f7348 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:03.262306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:03.269457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd85a27-a5c5-4ed7-b0d8-32885e5f7348 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:03.306734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a7c93-01df-4401-aa54-dd275448011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:03.307485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7c93-01df-4401-aa54-dd275448011e None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:03.307765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7c93-01df-4401-aa54-dd275448011e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:03.308039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a7c93-01df-4401-aa54-dd275448011e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:03.308739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a7c93-01df-4401-aa54-dd275448011e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:03.310354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a7c93-01df-4401-aa54-dd275448011e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:03.336482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 505/1020] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:02:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:02:04.521900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374874a-8bf7-4de2-8558-feb3e0b21b79 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:04.522412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374874a-8bf7-4de2-8558-feb3e0b21b79 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:04.522412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374874a-8bf7-4de2-8558-feb3e0b21b79 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:04.523917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 516/1021] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:04.529246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d33f-3bef-4198-8944-1d0331b39d42 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:04.529695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d33f-3bef-4198-8944-1d0331b39d42 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:04.529791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d33f-3bef-4198-8944-1d0331b39d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:04.548150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4d56d33f-3bef-4198-8944-1d0331b39d42 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:04.549349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4d56d33f-3bef-4198-8944-1d0331b39d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:04.627317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 506/1022] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:04.652301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374874a-8bf7-4de2-8558-feb3e0b21b79 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:04.728488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:04.761928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374874a-8bf7-4de2-8558-feb3e0b21b79 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:04.802685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1923-811f-43af-a6f9-905ed948a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:04.803323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1923-811f-43af-a6f9-905ed948a094 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:04.803587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1923-811f-43af-a6f9-905ed948a094 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:04.803856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1923-811f-43af-a6f9-905ed948a094 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:04.804969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb1923-811f-43af-a6f9-905ed948a094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:04.806431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb1923-811f-43af-a6f9-905ed948a094 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:04.812778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 517/1023] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:04.813943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56d33f-3bef-4198-8944-1d0331b39d42 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:04.893320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:04.919373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56d33f-3bef-4198-8944-1d0331b39d42 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:04.958537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854b289-ad95-4a97-bd13-a7ff6867af06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:04.960861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854b289-ad95-4a97-bd13-a7ff6867af06 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:04.961364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854b289-ad95-4a97-bd13-a7ff6867af06 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:04.963812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854b289-ad95-4a97-bd13-a7ff6867af06 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:04.964538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854b289-ad95-4a97-bd13-a7ff6867af06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:04.965982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854b289-ad95-4a97-bd13-a7ff6867af06 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:04.996506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 507/1024] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:02:04 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:02:05.142109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8674be-02c5-47a0-951d-97d885ffae4f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.142471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8674be-02c5-47a0-951d-97d885ffae4f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.142664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8674be-02c5-47a0-951d-97d885ffae4f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.143998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 518/1025] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:05.149441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131305-6ebe-4474-b009-162e49d6a37d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.149719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131305-6ebe-4474-b009-162e49d6a37d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.149995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131305-6ebe-4474-b009-162e49d6a37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.168645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-84131305-6ebe-4474-b009-162e49d6a37d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:05.169914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-84131305-6ebe-4474-b009-162e49d6a37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:05.242510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 508/1026] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:05.266441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8674be-02c5-47a0-951d-97d885ffae4f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:05.373914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8674be-02c5-47a0-951d-97d885ffae4f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:05.414275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f19bf1-c9b0-48ea-a031-fabdf779e40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:05.415449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f19bf1-c9b0-48ea-a031-fabdf779e40c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.415745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f19bf1-c9b0-48ea-a031-fabdf779e40c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.415959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f19bf1-c9b0-48ea-a031-fabdf779e40c None admin] PATH_INFO: `/v3/services/image` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.416541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f19bf1-c9b0-48ea-a031-fabdf779e40c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:05.417899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f19bf1-c9b0-48ea-a031-fabdf779e40c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c4f19bf1-c9b0-48ea-a031-fabdf779e40c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:05.426709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:05.427666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:05.428550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 519/1027] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Aug 9 19:02:05 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:05.436432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131305-6ebe-4474-b009-162e49d6a37d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:05.500371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:05.523030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7620ca-cdda-496d-9a03-95433d47a99a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.523534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7620ca-cdda-496d-9a03-95433d47a99a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.523645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7620ca-cdda-496d-9a03-95433d47a99a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.525330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 520/1028] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:05.534201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f299bb-92d4-4e9e-8bda-f939973afaf8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.534474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f299bb-92d4-4e9e-8bda-f939973afaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.534743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f299bb-92d4-4e9e-8bda-f939973afaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.542129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:05.543981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131305-6ebe-4474-b009-162e49d6a37d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:05.561906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-38f299bb-92d4-4e9e-8bda-f939973afaf8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:05.563100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-38f299bb-92d4-4e9e-8bda-f939973afaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:05.586439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:05.587233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.587518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.587796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.588429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] RBAC: Adding query filter params (name=image) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:05.588764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:05.590117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:05.596412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 509/1029] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:05.601869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:05.643279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 521/1030] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:05.666991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f299bb-92d4-4e9e-8bda-f939973afaf8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:05.709225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a07f82-8b79-40a1-8c17-5f296edadf9b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:05.740028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:05.750720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:05.751654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.751940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.752208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.752876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:05.753177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:05.754563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:05.759942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 510/1031] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:05.765696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:05.776023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f299bb-92d4-4e9e-8bda-f939973afaf8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:05.807947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:05.814247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97f548-25b2-4883-94e6-34271fdf12f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:05.814877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97f548-25b2-4883-94e6-34271fdf12f4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.815155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97f548-25b2-4883-94e6-34271fdf12f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.815428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97f548-25b2-4883-94e6-34271fdf12f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.826973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97f548-25b2-4883-94e6-34271fdf12f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5a97f548-25b2-4883-94e6-34271fdf12f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:05.832075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:05.833063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:05.833887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 522/1032] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Aug 9 19:02:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:05.839754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97f548-25b2-4883-94e6-34271fdf12f4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:05.880305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935928e-fb08-4787-8d9a-30515834cfd5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:05.918532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:05.919256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.920614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.920614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.920614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] RBAC: Adding query filter params (interface=public, service_id=97be80595cda454a83ceed09a3fcfe6f, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:05.920962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:05.922298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:05.932806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 511/1033] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Aug 9 19:02:05 2022] GET /identity/v3/endpoints?service_id=97be80595cda454a83ceed09a3fcfe6f&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:05.933695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:05.945844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97f548-25b2-4883-94e6-34271fdf12f4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:05.983436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:05.984044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:05.984326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:05.984630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:05.985304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:05.985663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:05.987083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:05.997779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 523/1034] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Aug 9 19:02:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:06.003362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:06.006906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:06.061572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29909c42-a7c1-4d31-8370-dc08b9d876c7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:06.064006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:06.106661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d59da-217a-407b-8b4f-616d8189990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:06.108146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d59da-217a-407b-8b4f-616d8189990e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:06.108698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d59da-217a-407b-8b4f-616d8189990e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:06.109310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d59da-217a-407b-8b4f-616d8189990e None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:06.110455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d59da-217a-407b-8b4f-616d8189990e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:06.112471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d59da-217a-407b-8b4f-616d8189990e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:06.115623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6ee00-410f-4566-bc67-25da4695ab9b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:06.122463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 512/1035] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:02:05 2022] GET /identity/v3/services => generated 1957 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:02:06.171581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:06.171581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:06.171906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:06.172719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:06.187674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:06.189247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:02:06.207133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:06.208165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:02:06.209107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:02:06.209107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:06.209107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:06.209107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 524/1036] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:06.214865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d59da-217a-407b-8b4f-616d8189990e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:06.343165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d59da-217a-407b-8b4f-616d8189990e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:06.365990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:06.382310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:06.384486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:06.384486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:06.384486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:06.384486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:06.384794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:06.386320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:06.401413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 513/1037] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:06.407179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:06.521122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2936d-f12e-409f-9a9e-4ff457026ae0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:06.529427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:06.563261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:06.563895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:06.564171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:06.564462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:06.573822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:06.575285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:02:06.588715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:06.589751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:06.590701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 525/1038] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:02:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:06.597532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:06.709665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a9e2f-47df-4fbd-855e-59f1273f2fd9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:06.752092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:06.752725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:06.753000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:06.753265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:06.753987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:06.754289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:06.755693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:06.773737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 514/1039] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:02:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:06.780587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:06.883154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91b4d6-4fe6-42c9-97a5-c4aa5954e503 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:06.918486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b18c82-60e4-49cd-bd41-a923862f45b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:06.919091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18c82-60e4-49cd-bd41-a923862f45b1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:06.919370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18c82-60e4-49cd-bd41-a923862f45b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:06.919637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18c82-60e4-49cd-bd41-a923862f45b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:06.954150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:06.989835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b18c82-60e4-49cd-bd41-a923862f45b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:06.991282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b18c82-60e4-49cd-bd41-a923862f45b1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:07.105197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 526/1040] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:06 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:07.111109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.140673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e060c27-a209-4e9b-b1f7-1e2bce70f070 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.140882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e060c27-a209-4e9b-b1f7-1e2bce70f070 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.141158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e060c27-a209-4e9b-b1f7-1e2bce70f070 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.142625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 527/1041] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:07.147383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4fabe-0f8d-477f-83fe-82cd20f1853e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.147670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4fabe-0f8d-477f-83fe-82cd20f1853e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.147949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4fabe-0f8d-477f-83fe-82cd20f1853e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.156244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:07.166195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7ac4fabe-0f8d-477f-83fe-82cd20f1853e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:07.167390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ac4fabe-0f8d-477f-83fe-82cd20f1853e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:07.221796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8decad7-d726-4348-a008-ee95757b3ea8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.235046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:07.244677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 528/1042] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:07.265040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.265702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.265982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.266272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.267419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4fabe-0f8d-477f-83fe-82cd20f1853e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.279754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:07.281614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:02:07.299377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:07.300427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:02:07.308070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:02:07.308070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:07.308070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:07.308070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 515/1043] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:07.308070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.392111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4fabe-0f8d-477f-83fe-82cd20f1853e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.417964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:07.428893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb15e7-7c54-41ab-aa54-934cf7c0da9f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.430014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72451b12-ad8f-499b-a138-74bfa3221d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.430743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451b12-ad8f-499b-a138-74bfa3221d03 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.431025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451b12-ad8f-499b-a138-74bfa3221d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.431297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451b12-ad8f-499b-a138-74bfa3221d03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.441941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72451b12-ad8f-499b-a138-74bfa3221d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-72451b12-ad8f-499b-a138-74bfa3221d03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:07.446607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:07.448823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:07.449879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 529/1044] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:07.454017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72451b12-ad8f-499b-a138-74bfa3221d03 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.477196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.479019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.479019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.479254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.479909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:07.480216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:07.481618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:07.500427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 516/1045] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:07.501465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.519793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:07.573089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72451b12-ad8f-499b-a138-74bfa3221d03 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.606667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a1340-9aa4-46e4-9b37-9900d60bdd93 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.610266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.610867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.611146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.611416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.612082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:07.612383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:07.613783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:07.628217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 530/1046] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:07.629276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.648625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.649941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.650211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.650762 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.659322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:07.661262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:02:07.673705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:07.674618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:07.675626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:02:07.675626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:02:07.675626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:02:07.675626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:02:07.675626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:02:07.675626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:02:07.675626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:02:07.675626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:07.676826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 517/1047] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:07.681701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.695024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:07.744014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5a995-b3a4-462d-8ded-381e9a21161a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.762476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:07.781196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb4b23-d948-4f00-8f69-3aee6de370c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.782059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb4b23-d948-4f00-8f69-3aee6de370c4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.782348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb4b23-d948-4f00-8f69-3aee6de370c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.782627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb4b23-d948-4f00-8f69-3aee6de370c4 None admin] PATH_INFO: `/v3/users/05b867dde1504099948b1dd58b7a210d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.793804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdfb4-dd50-4386-9aaa-9adfd351fd95 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.798332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb4b23-d948-4f00-8f69-3aee6de370c4 None admin] RBAC: Authorizing `identity:get_user(user_id=05b867dde1504099948b1dd58b7a210d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:07.799788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb4b23-d948-4f00-8f69-3aee6de370c4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:07.801516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 531/1048] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/users/05b867dde1504099948b1dd58b7a210d => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:07.807956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb4b23-d948-4f00-8f69-3aee6de370c4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.836393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.837687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.837967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.838242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.839030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:07.839338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:07.840786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:07.853109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 518/1049] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:07.858841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:07.919750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb4b23-d948-4f00-8f69-3aee6de370c4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:07.933679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:07.958496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52391e7a-9f3e-4e92-8792-b1bd741ab469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:07.959433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391e7a-9f3e-4e92-8792-b1bd741ab469 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:07.960044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391e7a-9f3e-4e92-8792-b1bd741ab469 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:07.960329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391e7a-9f3e-4e92-8792-b1bd741ab469 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:07.968466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781c758-82fe-4825-9105-d9272881e6be None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:08.004321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:08.004982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:08.005262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:08.005535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:08.006256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] RBAC: Adding query filter params (role.id=8e5e78234e364f77a4017d19ebe64dc0, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=48410d57fe0d489cbb07cfca58ca7b30) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:08.006567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:08.007982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:08.016485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 519/1050] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/role_assignments?user.id=48410d57fe0d489cbb07cfca58ca7b30&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=8e5e78234e364f77a4017d19ebe64dc0 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:08.023968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:08.052619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391e7a-9f3e-4e92-8792-b1bd741ab469 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:08.054105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391e7a-9f3e-4e92-8792-b1bd741ab469 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:08.168834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 532/1051] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 19:02:08.174628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:08.219362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527599e2-704f-44a3-9fc4-c634c8bd245c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:08.219692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527599e2-704f-44a3-9fc4-c634c8bd245c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:08.219902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527599e2-704f-44a3-9fc4-c634c8bd245c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:08.221400 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 533/1052] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:02:08.226152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ef54d-b73b-4fe3-a285-b2ddaa8fef40 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:08.226432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ef54d-b73b-4fe3-a285-b2ddaa8fef40 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:08.226705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ef54d-b73b-4fe3-a285-b2ddaa8fef40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:08.245268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4d4ef54d-b73b-4fe3-a285-b2ddaa8fef40 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:08.246436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4d4ef54d-b73b-4fe3-a285-b2ddaa8fef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:08.287208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b721ea-2e22-46fd-9fa0-732d91acef86 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:08.324130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 534/1053] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:08.324644 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd3f37-3d01-428d-8107-a2d9c65da228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:08.325281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd3f37-3d01-428d-8107-a2d9c65da228 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:08.325566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd3f37-3d01-428d-8107-a2d9c65da228 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:08.325835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd3f37-3d01-428d-8107-a2d9c65da228 None admin] PATH_INFO: `/v3/projects/20c05749a352455a98813755480ce1d8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:08.326965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd3f37-3d01-428d-8107-a2d9c65da228 None admin] RBAC: Authorizing `identity:get_project(project_id=20c05749a352455a98813755480ce1d8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:08.328373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd3f37-3d01-428d-8107-a2d9c65da228 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:08.334827 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 520/1054] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:08 2022] GET /identity/v3/projects/20c05749a352455a98813755480ce1d8 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:08.336239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ef54d-b73b-4fe3-a285-b2ddaa8fef40 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:08.346802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:08.352860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd3f37-3d01-428d-8107-a2d9c65da228 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:08.356229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:08.464386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ef54d-b73b-4fe3-a285-b2ddaa8fef40 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:08.476699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd3f37-3d01-428d-8107-a2d9c65da228 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:08.505002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:08.505614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:08.505886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:08.506158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:08.506884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=20c05749a352455a98813755480ce1d8, user.id=05b867dde1504099948b1dd58b7a210d) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:08.507194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:08.508678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:08.514586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:08.514785 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 535/1055] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:08 2022] GET /identity/v3/role_assignments?user.id=05b867dde1504099948b1dd58b7a210d&scope.project.id=20c05749a352455a98813755480ce1d8&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:08.515618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:08.515903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:08.516173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] PATH_INFO: `/v3/services/image` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:08.516788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:08.518170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:08.534211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:08.535309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:08.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 521/1056] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Aug 9 19:02:08 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:08.545833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:08.621516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:08.656831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c396e-d97e-48a3-9c51-6e9fddaa264c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:08.693839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:08.694443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:08.694716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:08.694983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:08.695602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] RBAC: Adding query filter params (name=image) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:08.695935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:08.697626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:08.702682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 536/1057] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:08.708322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:08.779508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:08.818377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a240d-e9b1-4060-97e1-61ea9b5b4c83 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:08.852345 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eb21e-2c9e-4aa7-894a-1ba5357d5777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:08.853023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb21e-2c9e-4aa7-894a-1ba5357d5777 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:08.853263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb21e-2c9e-4aa7-894a-1ba5357d5777 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:08.853591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb21e-2c9e-4aa7-894a-1ba5357d5777 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:08.854209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb21e-2c9e-4aa7-894a-1ba5357d5777 None admin] RBAC: Adding query filter params (type=image) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:08.854541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb21e-2c9e-4aa7-894a-1ba5357d5777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:08.855887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb21e-2c9e-4aa7-894a-1ba5357d5777 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:08.860909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 522/1058] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:08.866473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:08.966910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430be0f7-a07d-46dd-b6bc-2a3677c6b36d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:09.000617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf70fca-9c13-4f2a-b820-083520c889fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:09.001215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf70fca-9c13-4f2a-b820-083520c889fe None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:09.001488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf70fca-9c13-4f2a-b820-083520c889fe None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:09.001755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf70fca-9c13-4f2a-b820-083520c889fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:09.002411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf70fca-9c13-4f2a-b820-083520c889fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:09.003853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf70fca-9c13-4f2a-b820-083520c889fe None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:09.008369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:09.026328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 537/1059] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:02:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:02:10.105920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be56acd-1bd3-42c9-b1be-e146932fe358 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.106440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be56acd-1bd3-42c9-b1be-e146932fe358 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.106440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be56acd-1bd3-42c9-b1be-e146932fe358 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.107842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 523/1060] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:10.113263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84ea9-f9a9-491b-9d2f-3b0f44468d0c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.113670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84ea9-f9a9-491b-9d2f-3b0f44468d0c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.113853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84ea9-f9a9-491b-9d2f-3b0f44468d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.133757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c6b84ea9-f9a9-491b-9d2f-3b0f44468d0c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:10.134968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c6b84ea9-f9a9-491b-9d2f-3b0f44468d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:10.212827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 538/1061] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:10.235045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be56acd-1bd3-42c9-b1be-e146932fe358 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:10.338900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be56acd-1bd3-42c9-b1be-e146932fe358 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:10.367335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:10.374725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2961ea-08ce-4db2-9e50-ad58e6673282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:10.375354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2961ea-08ce-4db2-9e50-ad58e6673282 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.375633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2961ea-08ce-4db2-9e50-ad58e6673282 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.375902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2961ea-08ce-4db2-9e50-ad58e6673282 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.393863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2961ea-08ce-4db2-9e50-ad58e6673282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-8e2961ea-08ce-4db2-9e50-ad58e6673282 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:10.398510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:10.399500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:02:10.400322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:10.404845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 524/1062] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Aug 9 19:02:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:10.405687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b84ea9-f9a9-491b-9d2f-3b0f44468d0c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:10.506301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b84ea9-f9a9-491b-9d2f-3b0f44468d0c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:10.524051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:10.544228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630fe64-ae73-464b-bb9e-9a79aa0af054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:10.545013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630fe64-ae73-464b-bb9e-9a79aa0af054 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.545368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630fe64-ae73-464b-bb9e-9a79aa0af054 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.545643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630fe64-ae73-464b-bb9e-9a79aa0af054 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.546310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630fe64-ae73-464b-bb9e-9a79aa0af054 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:10.546613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630fe64-ae73-464b-bb9e-9a79aa0af054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:10.547981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630fe64-ae73-464b-bb9e-9a79aa0af054 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:10.561861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 539/1063] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Aug 9 19:02:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:10.563064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2961ea-08ce-4db2-9e50-ad58e6673282 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:10.566205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04b3e8-a72e-4991-b181-a0ab441f7cb8 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.566395 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04b3e8-a72e-4991-b181-a0ab441f7cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.566818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04b3e8-a72e-4991-b181-a0ab441f7cb8 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.568476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 540/1064] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:02:10.573648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8d607-c957-4272-a4ec-7b25718eea7a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.574046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8d607-c957-4272-a4ec-7b25718eea7a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.574322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8d607-c957-4272-a4ec-7b25718eea7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.592884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0df8d607-c957-4272-a4ec-7b25718eea7a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:10.594054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0df8d607-c957-4272-a4ec-7b25718eea7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:10.664011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 541/1065] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:10.669345 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2961ea-08ce-4db2-9e50-ad58e6673282 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:10.686673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8d607-c957-4272-a4ec-7b25718eea7a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:10.706233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:10.706868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.707148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.707437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.718641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:10.720085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:02:10.735307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:10.736280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:02:10.737265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:02:10.737265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:10.737265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:10.738887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 525/1066] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:10.744452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:10.791828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:10.797983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8d607-c957-4272-a4ec-7b25718eea7a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:10.807555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:10.835659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9be068-b5ce-4166-addb-89e03031724f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:10.836293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be068-b5ce-4166-addb-89e03031724f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.837193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be068-b5ce-4166-addb-89e03031724f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.837466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be068-b5ce-4166-addb-89e03031724f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.847737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9be068-b5ce-4166-addb-89e03031724f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9f9be068-b5ce-4166-addb-89e03031724f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:10.852426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:10.853752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:10.854612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 542/1067] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:10.859491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9be068-b5ce-4166-addb-89e03031724f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:10.862372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56670e10-1566-4709-b857-518f731f4da4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:10.899682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:10.900278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:10.900580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:10.900848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:10.901510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:10.901808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:10.903155 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:10.915691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 526/1068] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:10.920845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:10.970087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9be068-b5ce-4166-addb-89e03031724f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.007131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.008127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.009190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.009468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.010132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:11.010436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.011803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.020924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 543/1069] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:11.027094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.028642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2732-4fcc-4b22-9b85-4789eba00bf2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.030397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:11.061940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:11.074295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472bb95-e2c7-4247-9b88-93bee2e335f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.074972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472bb95-e2c7-4247-9b88-93bee2e335f7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.075200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472bb95-e2c7-4247-9b88-93bee2e335f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.075893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472bb95-e2c7-4247-9b88-93bee2e335f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.084018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472bb95-e2c7-4247-9b88-93bee2e335f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.085657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472bb95-e2c7-4247-9b88-93bee2e335f7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f472bb95-e2c7-4247-9b88-93bee2e335f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:02:11.098558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:11.099728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:11.100702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 527/1070] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:02:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:11.101296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eeae1-899b-4aad-86ad-2f8fe4930eae None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.101491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eeae1-899b-4aad-86ad-2f8fe4930eae None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.101759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eeae1-899b-4aad-86ad-2f8fe4930eae None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.103004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 528/1071] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:11.107772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873eeae1-899b-4aad-86ad-2f8fe4930eae None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.142241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e2139-7200-47b4-9212-7afa052e576b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.175761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9596e6-0c8d-498a-af51-57bc58f7f587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.176303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9596e6-0c8d-498a-af51-57bc58f7f587 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.176609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9596e6-0c8d-498a-af51-57bc58f7f587 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.176880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9596e6-0c8d-498a-af51-57bc58f7f587 None admin] PATH_INFO: `/v3/users/05b867dde1504099948b1dd58b7a210d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.177983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9596e6-0c8d-498a-af51-57bc58f7f587 None admin] RBAC: Authorizing `identity:get_user(user_id=05b867dde1504099948b1dd58b7a210d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.179403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9596e6-0c8d-498a-af51-57bc58f7f587 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.181866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 544/1072] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/users/05b867dde1504099948b1dd58b7a210d => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:11.183054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37652d2-95c9-4d8c-ab49-68fad0fc9433 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.183442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37652d2-95c9-4d8c-ab49-68fad0fc9433 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.183650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37652d2-95c9-4d8c-ab49-68fad0fc9433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.203163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c37652d2-95c9-4d8c-ab49-68fad0fc9433 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:11.204314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c37652d2-95c9-4d8c-ab49-68fad0fc9433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:11.214083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873eeae1-899b-4aad-86ad-2f8fe4930eae None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.247838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.248463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.248733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.249002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.249721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:11.250021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.251431 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.268534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 529/1073] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:11.269340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.275937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 545/1074] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:11.276786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37652d2-95c9-4d8c-ab49-68fad0fc9433 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.293932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:11.311041 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:11.374974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465267f4-add7-4f06-969f-00262bc95763 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.381409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37652d2-95c9-4d8c-ab49-68fad0fc9433 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.409446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf8ad-eda3-47e3-97f5-a024a1a8f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.410075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf8ad-eda3-47e3-97f5-a024a1a8f722 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.410348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf8ad-eda3-47e3-97f5-a024a1a8f722 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.410612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edf8ad-eda3-47e3-97f5-a024a1a8f722 None admin] PATH_INFO: `/v3/projects/20c05749a352455a98813755480ce1d8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.411702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edf8ad-eda3-47e3-97f5-a024a1a8f722 None admin] RBAC: Authorizing `identity:get_project(project_id=20c05749a352455a98813755480ce1d8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.413149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edf8ad-eda3-47e3-97f5-a024a1a8f722 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.415317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 530/1075] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/projects/20c05749a352455a98813755480ce1d8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:11.416056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf8ad-eda3-47e3-97f5-a024a1a8f722 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.420889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50968060-3193-4609-a297-59f64600895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.421576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50968060-3193-4609-a297-59f64600895a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.421848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50968060-3193-4609-a297-59f64600895a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.422109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50968060-3193-4609-a297-59f64600895a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.433578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50968060-3193-4609-a297-59f64600895a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.435208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50968060-3193-4609-a297-59f64600895a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-50968060-3193-4609-a297-59f64600895a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:02:11.450283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:11.451301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:02:11.452187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:02:11.452187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:11.452187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:11.452187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 546/1076] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:11.452407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50968060-3193-4609-a297-59f64600895a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.513789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:11.528271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edf8ad-eda3-47e3-97f5-a024a1a8f722 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.532206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:11.551772 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50968060-3193-4609-a297-59f64600895a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.564390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682c428-e3a5-4d74-849c-8e505846eb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.565026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682c428-e3a5-4d74-849c-8e505846eb0f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.565306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682c428-e3a5-4d74-849c-8e505846eb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.565576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682c428-e3a5-4d74-849c-8e505846eb0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.590659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1358063-acec-4a94-8023-a67e3ca08f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.591370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1358063-acec-4a94-8023-a67e3ca08f6d None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.591648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1358063-acec-4a94-8023-a67e3ca08f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.591977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1358063-acec-4a94-8023-a67e3ca08f6d None admin] PATH_INFO: `/v3/projects/20c05749a352455a98813755480ce1d8/users/05b867dde1504099948b1dd58b7a210d/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.593948 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1358063-acec-4a94-8023-a67e3ca08f6d None admin] RBAC: Authorizing `identity:create_grant(project_id=20c05749a352455a98813755480ce1d8, user_id=05b867dde1504099948b1dd58b7a210d, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.595370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1358063-acec-4a94-8023-a67e3ca08f6d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.608450 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 547/1077] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:02:11 2022] PUT /identity/v3/projects/20c05749a352455a98813755480ce1d8/users/05b867dde1504099948b1dd58b7a210d/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:02:11.609272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1358063-acec-4a94-8023-a67e3ca08f6d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.639580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682c428-e3a5-4d74-849c-8e505846eb0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.641222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682c428-e3a5-4d74-849c-8e505846eb0f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.746537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1358063-acec-4a94-8023-a67e3ca08f6d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.750431 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:11.765296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 531/1078] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:11.771827 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682c428-e3a5-4d74-849c-8e505846eb0f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.791530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.792140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.792418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.792716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.793422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:11.793725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.795162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.804738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:11.812778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 548/1079] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:11.814710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.890571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682c428-e3a5-4d74-849c-8e505846eb0f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.922496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf948-a673-4187-8917-bdf3ac6158c1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:11.927393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.928032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.928309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.928631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.940617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.942063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:02:11.958108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:11.959107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:02:11.959982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:02:11.959982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:11.959982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:11.959982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 532/1080] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:11.965231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:11.967055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:11.967411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:11.967687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:11.967954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:11.975114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:11.976596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:02:11.988640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:11.989536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:11.990428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:02:11.990428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:02:11.990428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:02:11.990428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:02:11.990428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:02:11.990428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:02:11.990428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:02:11.990428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:11.993320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 549/1081] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:11.995893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:12.015092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:12.063357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:12.081179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b900e2-83b0-4321-aedc-02b989e4a940 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:12.110123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4792-58c8-4ff4-8ab9-80206d718f15 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:12.117067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:12.118168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:12.118168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:12.118365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:12.118965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:12.119264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:12.120634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:12.132767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 533/1082] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:12.147033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade5262-7d94-4a8a-b546-43eb8884eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:12.147718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5262-7d94-4a8a-b546-43eb8884eee1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:12.147932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5262-7d94-4a8a-b546-43eb8884eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:12.148203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5262-7d94-4a8a-b546-43eb8884eee1 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:12.149006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade5262-7d94-4a8a-b546-43eb8884eee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:12.149829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade5262-7d94-4a8a-b546-43eb8884eee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:12.151305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade5262-7d94-4a8a-b546-43eb8884eee1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:12.165918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 550/1083] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:02:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:12.171983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:12.282946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:12.287047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206995d8-a99f-4073-98cc-716b325a17fe None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:12.331294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083d9f7-0716-4fee-8044-b4f3336d742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:12.332326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d9f7-0716-4fee-8044-b4f3336d742d None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:12.332892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d9f7-0716-4fee-8044-b4f3336d742d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:12.333172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083d9f7-0716-4fee-8044-b4f3336d742d None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/48410d57fe0d489cbb07cfca58ca7b30/roles/8e5e78234e364f77a4017d19ebe64dc0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:12.335429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083d9f7-0716-4fee-8044-b4f3336d742d None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=48410d57fe0d489cbb07cfca58ca7b30, role_id=8e5e78234e364f77a4017d19ebe64dc0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:12.337878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083d9f7-0716-4fee-8044-b4f3336d742d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:12.354262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 534/1084] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:02:12 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/48410d57fe0d489cbb07cfca58ca7b30/roles/8e5e78234e364f77a4017d19ebe64dc0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:02:13.645813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f7d0-cab5-46d6-8faa-d81da5897786 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:13.646312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f7d0-cab5-46d6-8faa-d81da5897786 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:13.646312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f7d0-cab5-46d6-8faa-d81da5897786 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:13.647596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 551/1085] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:13.652875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee9725-d617-4377-bd41-743ec13c4c40 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:13.653287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee9725-d617-4377-bd41-743ec13c4c40 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:13.653422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee9725-d617-4377-bd41-743ec13c4c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:13.671791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-dcee9725-d617-4377-bd41-743ec13c4c40 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:13.673007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dcee9725-d617-4377-bd41-743ec13c4c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:13.737109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 535/1086] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:13.760384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00f7d0-cab5-46d6-8faa-d81da5897786 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:13.860290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00f7d0-cab5-46d6-8faa-d81da5897786 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:13.882016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:13.897200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e1ffe-8df8-45a6-aa2f-8e586b753d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:13.897749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e1ffe-8df8-45a6-aa2f-8e586b753d40 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:13.898024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e1ffe-8df8-45a6-aa2f-8e586b753d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:13.898291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e1ffe-8df8-45a6-aa2f-8e586b753d40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:13.908125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e1ffe-8df8-45a6-aa2f-8e586b753d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b91e1ffe-8df8-45a6-aa2f-8e586b753d40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:13.912736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:13.913685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:13.914847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 552/1087] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:13.920011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee9725-d617-4377-bd41-743ec13c4c40 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:14.010091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:14.021610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee9725-d617-4377-bd41-743ec13c4c40 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:14.055988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:14.056786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.057069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.057346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.058017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:14.058323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:14.060023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:14.067932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 536/1088] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:14.074745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e1ffe-8df8-45a6-aa2f-8e586b753d40 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:14.174302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e1ffe-8df8-45a6-aa2f-8e586b753d40 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:14.209374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffe854-14f1-4206-a7b8-125133e94da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:14.209998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe854-14f1-4206-a7b8-125133e94da4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.210275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe854-14f1-4206-a7b8-125133e94da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.210544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe854-14f1-4206-a7b8-125133e94da4 None admin] PATH_INFO: `/v3/users/05b867dde1504099948b1dd58b7a210d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.211822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe854-14f1-4206-a7b8-125133e94da4 None admin] RBAC: Authorizing `identity:get_user(user_id=05b867dde1504099948b1dd58b7a210d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:14.213426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe854-14f1-4206-a7b8-125133e94da4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:14.215001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 553/1089] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:14 2022] GET /identity/v3/users/05b867dde1504099948b1dd58b7a210d => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:14.220990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:14.329106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9d4cb-ced6-40c1-9f7d-705c638cab4d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:14.331940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3c120-f208-4221-ba2e-d3687b4a607e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.332224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3c120-f208-4221-ba2e-d3687b4a607e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.332520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3c120-f208-4221-ba2e-d3687b4a607e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.333847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 554/1090] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:14.338533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb064ce4-8686-443b-8a76-2e05c8028337 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.338813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb064ce4-8686-443b-8a76-2e05c8028337 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.339085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb064ce4-8686-443b-8a76-2e05c8028337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.357082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fb064ce4-8686-443b-8a76-2e05c8028337 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:14.358096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fb064ce4-8686-443b-8a76-2e05c8028337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:14.364113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bc63d-9778-4b5e-bceb-3535059b0ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:14.364747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bc63d-9778-4b5e-bceb-3535059b0ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.365028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bc63d-9778-4b5e-bceb-3535059b0ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.365300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bc63d-9778-4b5e-bceb-3535059b0ec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.388007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:14.407315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:14.431627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 555/1091] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:14.436911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bc63d-9778-4b5e-bceb-3535059b0ec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:14.438417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bc63d-9778-4b5e-bceb-3535059b0ec2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:14.454895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb064ce4-8686-443b-8a76-2e05c8028337 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:14.534812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 537/1092] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:14.541528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bc63d-9778-4b5e-bceb-3535059b0ec2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:14.555810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb064ce4-8686-443b-8a76-2e05c8028337 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:14.589647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702c3d-271c-4f03-b948-d4c956956236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:14.590266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702c3d-271c-4f03-b948-d4c956956236 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.590536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702c3d-271c-4f03-b948-d4c956956236 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.590805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702c3d-271c-4f03-b948-d4c956956236 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.594900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:14.601369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b702c3d-271c-4f03-b948-d4c956956236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8b702c3d-271c-4f03-b948-d4c956956236 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:14.606618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:14.607626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:14.613821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 556/1093] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Aug 9 19:02:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:14.614505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702c3d-271c-4f03-b948-d4c956956236 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:14.643926 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bc63d-9778-4b5e-bceb-3535059b0ec2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:14.654283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:14.678615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035b22-bdd9-4fd9-9c50-d05a9ce031f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:14.679236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035b22-bdd9-4fd9-9c50-d05a9ce031f9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.679508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035b22-bdd9-4fd9-9c50-d05a9ce031f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.679775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035b22-bdd9-4fd9-9c50-d05a9ce031f9 None admin] PATH_INFO: `/v3/projects/20c05749a352455a98813755480ce1d8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.681102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70035b22-bdd9-4fd9-9c50-d05a9ce031f9 None admin] RBAC: Authorizing `identity:get_project(project_id=20c05749a352455a98813755480ce1d8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:14.683078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70035b22-bdd9-4fd9-9c50-d05a9ce031f9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:14.684920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 538/1094] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:14 2022] GET /identity/v3/projects/20c05749a352455a98813755480ce1d8 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:14.690837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035b22-bdd9-4fd9-9c50-d05a9ce031f9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:14.714476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702c3d-271c-4f03-b948-d4c956956236 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:14.748690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:14.749281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.749553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.749820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.750468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:14.750766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:14.752121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:14.759912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 557/1095] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Aug 9 19:02:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:14.765885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:14.789409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035b22-bdd9-4fd9-9c50-d05a9ce031f9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:14.810084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:14.824758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:14.825360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.825633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.825900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.826611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=20c05749a352455a98813755480ce1d8, user.id=05b867dde1504099948b1dd58b7a210d) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:14.826920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:14.828322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:14.833942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 539/1096] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:14 2022] GET /identity/v3/role_assignments?user.id=05b867dde1504099948b1dd58b7a210d&scope.project.id=20c05749a352455a98813755480ce1d8&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:14.865676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:14.868909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db8baf-c513-49ef-8a7e-8a4a8a493276 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:14.906412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:14.907052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:14.907328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:14.907598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:14.920982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:14.922476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:02:14.937613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:14.938797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:02:14.939703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:02:14.939703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:14.939703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:14.939703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 558/1097] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:14.944583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:15.051057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4aa89f-4e73-4304-8c04-a1893dc00457 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:15.084385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:15.085011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:15.085290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:15.085564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:15.086202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:15.087153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:15.088574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:15.100220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 540/1098] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:15.106115 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:15.206781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabfb93-35e5-43c6-9dd3-76ad6e9d61bb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:15.239701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:15.240319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:15.240626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:15.240901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:15.248804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:15.250248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:02:15.261102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:15.262070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:15.263024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 559/1099] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Aug 9 19:02:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:15.267887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:15.313779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:15.369716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b38fb0-6f13-4a96-a547-dd50a1b1fd07 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:15.406443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:15.407053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:15.407339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:15.407738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:15.408322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:15.408652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:15.410070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:15.427804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 541/1100] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Aug 9 19:02:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:15.433814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:15.449096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:15.534553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1529afe-4e93-4440-9161-762423221f22 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:15.567758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459407db-1692-451f-9b52-4f14c4590382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:15.568358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459407db-1692-451f-9b52-4f14c4590382 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:15.568665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459407db-1692-451f-9b52-4f14c4590382 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:15.568936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459407db-1692-451f-9b52-4f14c4590382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:15.638065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459407db-1692-451f-9b52-4f14c4590382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:15.639501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459407db-1692-451f-9b52-4f14c4590382 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:15.653098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:15.742259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 560/1101] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:15.743462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:15.841250 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f3784-66ce-4f90-8795-ad57796d3f20 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:15.842686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:15.875818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:15.876460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:15.876761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:15.877033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:15.886947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:15.888370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:02:15.902297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:15.903277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:02:15.904161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:02:15.904161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:02:15.904161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:15.904161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 542/1102] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:02:15.909055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459407db-1692-451f-9b52-4f14c4590382 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:16.011348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459407db-1692-451f-9b52-4f14c4590382 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:16.025376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:16.046971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:16.047563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:16.047836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:16.048100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:16.048760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:16.049057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:16.050402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:16.061911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 561/1103] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:16.067467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:16.166565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d2c14-7344-491d-907b-dd231802d48d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:16.199753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:16.200375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:16.200683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:16.200958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:16.207391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:16.208847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:02:16.219675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:16.220743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:16.221935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:02:16.221935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:02:16.221935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:02:16.221935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:02:16.221935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:02:16.221935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:02:16.221935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:02:16.221935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:16.224588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 543/1104] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:02:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:16.226474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:16.324945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cda0aa-9c57-492b-961a-df2cf8f1e7bc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:16.357282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43370ca2-1e9d-479d-9e6e-ca6e71c84e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:16.357918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370ca2-1e9d-479d-9e6e-ca6e71c84e81 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:16.358194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370ca2-1e9d-479d-9e6e-ca6e71c84e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:16.358453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370ca2-1e9d-479d-9e6e-ca6e71c84e81 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:16.359134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43370ca2-1e9d-479d-9e6e-ca6e71c84e81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:16.359438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43370ca2-1e9d-479d-9e6e-ca6e71c84e81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:16.360890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43370ca2-1e9d-479d-9e6e-ca6e71c84e81 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:16.373512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 562/1105] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:16.379703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:16.398110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:16.482922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6af01c-bee2-4ca4-99cf-c8736d403e2e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:16.516537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d053e-c6a3-47c2-b0d8-45347f0f2cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:16.517132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d053e-c6a3-47c2-b0d8-45347f0f2cfb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:16.517408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d053e-c6a3-47c2-b0d8-45347f0f2cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:16.517675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d053e-c6a3-47c2-b0d8-45347f0f2cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:16.518386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d053e-c6a3-47c2-b0d8-45347f0f2cfb None admin] RBAC: Adding query filter params (role.id=8e5e78234e364f77a4017d19ebe64dc0, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=48410d57fe0d489cbb07cfca58ca7b30) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:16.518687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d053e-c6a3-47c2-b0d8-45347f0f2cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:16.520082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d053e-c6a3-47c2-b0d8-45347f0f2cfb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:16.526067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 544/1106] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:16 2022] GET /identity/v3/role_assignments?user.id=48410d57fe0d489cbb07cfca58ca7b30&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=8e5e78234e364f77a4017d19ebe64dc0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:16.875575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ab720-6e6f-4452-9128-343d2d5ee756 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:16.876054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ab720-6e6f-4452-9128-343d2d5ee756 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:16.876054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ab720-6e6f-4452-9128-343d2d5ee756 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:16.877518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 563/1107] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:16.882743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41189387-d9da-4a68-b5ba-0d5a2cdb2316 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:16.883444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41189387-d9da-4a68-b5ba-0d5a2cdb2316 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:16.883444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41189387-d9da-4a68-b5ba-0d5a2cdb2316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:16.903149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-41189387-d9da-4a68-b5ba-0d5a2cdb2316 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:16.904347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-41189387-d9da-4a68-b5ba-0d5a2cdb2316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:16.975698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 545/1108] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:17.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ab720-6e6f-4452-9128-343d2d5ee756 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:17.032336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:17.113047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ab720-6e6f-4452-9128-343d2d5ee756 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:17.148023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a3831-c2ec-44d1-96d9-bfde5edaf137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:17.148685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a3831-c2ec-44d1-96d9-bfde5edaf137 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:17.148995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a3831-c2ec-44d1-96d9-bfde5edaf137 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:17.149264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a3831-c2ec-44d1-96d9-bfde5edaf137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:17.150545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a3831-c2ec-44d1-96d9-bfde5edaf137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:17.151980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a3831-c2ec-44d1-96d9-bfde5edaf137 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:17.153614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 564/1109] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:17.159615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41189387-d9da-4a68-b5ba-0d5a2cdb2316 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:17.169987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:17.263096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41189387-d9da-4a68-b5ba-0d5a2cdb2316 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:17.298196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628c9b8-a1f0-4c51-b80e-012dc4bdbb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:17.298812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628c9b8-a1f0-4c51-b80e-012dc4bdbb74 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:17.299093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628c9b8-a1f0-4c51-b80e-012dc4bdbb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:17.299367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628c9b8-a1f0-4c51-b80e-012dc4bdbb74 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:17.300021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628c9b8-a1f0-4c51-b80e-012dc4bdbb74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:17.301499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628c9b8-a1f0-4c51-b80e-012dc4bdbb74 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:17.324608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 546/1110] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:02:17 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:02:19.405068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4398-aab6-43bd-a501-34b62d9f1664 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:19.405555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4398-aab6-43bd-a501-34b62d9f1664 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:19.405672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4398-aab6-43bd-a501-34b62d9f1664 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:19.407502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 565/1111] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:19.413047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7e21-4397-492e-a926-1b8627687854 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:19.413460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7e21-4397-492e-a926-1b8627687854 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:19.413601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a7e21-4397-492e-a926-1b8627687854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:19.432649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3b7a7e21-4397-492e-a926-1b8627687854 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:19.433847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3b7a7e21-4397-492e-a926-1b8627687854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:19.495180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:19.505785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 547/1112] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:19.532288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d4398-aab6-43bd-a501-34b62d9f1664 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:19.654383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:19.658994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d4398-aab6-43bd-a501-34b62d9f1664 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:19.694647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32847cce-10cb-4fa9-bd18-7fcaafc8f028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:19.695283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32847cce-10cb-4fa9-bd18-7fcaafc8f028 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:19.695564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32847cce-10cb-4fa9-bd18-7fcaafc8f028 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:19.695872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32847cce-10cb-4fa9-bd18-7fcaafc8f028 None admin] PATH_INFO: `/v3/roles/199fa373d2ff46b7a0c430b4a8c056e9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:19.697320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32847cce-10cb-4fa9-bd18-7fcaafc8f028 None admin] RBAC: Authorizing `identity:get_role(role_id=199fa373d2ff46b7a0c430b4a8c056e9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:19.698690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32847cce-10cb-4fa9-bd18-7fcaafc8f028 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:19.700178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 566/1113] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:19 2022] GET /identity/v3/roles/199fa373d2ff46b7a0c430b4a8c056e9 => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:19.705993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a7e21-4397-492e-a926-1b8627687854 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:19.806357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a7e21-4397-492e-a926-1b8627687854 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:19.841942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698a34a-da7e-40fc-aa84-c19af074962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:19.842578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698a34a-da7e-40fc-aa84-c19af074962b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:19.842858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698a34a-da7e-40fc-aa84-c19af074962b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:19.843130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698a34a-da7e-40fc-aa84-c19af074962b None admin] PATH_INFO: `/v3/users/be71e5d7713c42eeafa9734d86f5149c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:19.858264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698a34a-da7e-40fc-aa84-c19af074962b None admin] RBAC: Authorizing `identity:get_user(user_id=be71e5d7713c42eeafa9734d86f5149c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:19.859718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698a34a-da7e-40fc-aa84-c19af074962b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:19.861415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 548/1114] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:19 2022] GET /identity/v3/users/be71e5d7713c42eeafa9734d86f5149c => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:19.867244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32847cce-10cb-4fa9-bd18-7fcaafc8f028 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:19.965813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32847cce-10cb-4fa9-bd18-7fcaafc8f028 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:19.999063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fbc39-8de4-469a-bd3a-2a26011142fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:19.999659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fbc39-8de4-469a-bd3a-2a26011142fa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:19.999937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fbc39-8de4-469a-bd3a-2a26011142fa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:20.000207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fbc39-8de4-469a-bd3a-2a26011142fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:20.034343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:20.067169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fbc39-8de4-469a-bd3a-2a26011142fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:20.068650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fbc39-8de4-469a-bd3a-2a26011142fa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:20.168323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 567/1115] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:20.174480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698a34a-da7e-40fc-aa84-c19af074962b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:20.227639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:20.276243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698a34a-da7e-40fc-aa84-c19af074962b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:20.319137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee23e7-701c-4969-8aa6-127e24820ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:20.319773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee23e7-701c-4969-8aa6-127e24820ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:20.320049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee23e7-701c-4969-8aa6-127e24820ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:20.320336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee23e7-701c-4969-8aa6-127e24820ae9 None admin] PATH_INFO: `/v3/projects/20c05749a352455a98813755480ce1d8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:20.321532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee23e7-701c-4969-8aa6-127e24820ae9 None admin] RBAC: Authorizing `identity:get_project(project_id=20c05749a352455a98813755480ce1d8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:20.323001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee23e7-701c-4969-8aa6-127e24820ae9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:20.324640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 549/1116] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:20 2022] GET /identity/v3/projects/20c05749a352455a98813755480ce1d8 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:20.330820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fbc39-8de4-469a-bd3a-2a26011142fa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:20.399338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:20.434781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fbc39-8de4-469a-bd3a-2a26011142fa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:20.468447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb23b8-7d09-40be-9816-b66b00e960df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:20.469060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb23b8-7d09-40be-9816-b66b00e960df None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:20.469331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb23b8-7d09-40be-9816-b66b00e960df None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:20.469609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb23b8-7d09-40be-9816-b66b00e960df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:20.470441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb23b8-7d09-40be-9816-b66b00e960df None admin] RBAC: Adding query filter params (role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=20c05749a352455a98813755480ce1d8, user.id=be71e5d7713c42eeafa9734d86f5149c) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:20.470743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb23b8-7d09-40be-9816-b66b00e960df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:20.472158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb23b8-7d09-40be-9816-b66b00e960df None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:20.478057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 568/1117] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:20 2022] GET /identity/v3/role_assignments?user.id=be71e5d7713c42eeafa9734d86f5149c&scope.project.id=20c05749a352455a98813755480ce1d8&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:22.504954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c538d-f890-4f38-a83e-45e66f77f545 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:22.505490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c538d-f890-4f38-a83e-45e66f77f545 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:22.505490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c538d-f890-4f38-a83e-45e66f77f545 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:22.506789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 550/1118] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:22.512953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd7f0e-dc84-4d15-ba19-1de2b19ab8b0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:22.513383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd7f0e-dc84-4d15-ba19-1de2b19ab8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:22.513497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd7f0e-dc84-4d15-ba19-1de2b19ab8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:22.531654 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-16fd7f0e-dc84-4d15-ba19-1de2b19ab8b0 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:22.532880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-16fd7f0e-dc84-4d15-ba19-1de2b19ab8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:22.598236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 569/1119] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:22.621533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c538d-f890-4f38-a83e-45e66f77f545 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:22.721523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c538d-f890-4f38-a83e-45e66f77f545 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:22.741312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:22.755253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971604f-64c8-4425-8aa4-c026f9419313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:22.755868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971604f-64c8-4425-8aa4-c026f9419313 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:22.756141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971604f-64c8-4425-8aa4-c026f9419313 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:22.756410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971604f-64c8-4425-8aa4-c026f9419313 None admin] PATH_INFO: `/v3/roles/199fa373d2ff46b7a0c430b4a8c056e9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:22.757801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971604f-64c8-4425-8aa4-c026f9419313 None admin] RBAC: Authorizing `identity:get_role(role_id=199fa373d2ff46b7a0c430b4a8c056e9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:22.759773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971604f-64c8-4425-8aa4-c026f9419313 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:22.761081 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 551/1120] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:22 2022] GET /identity/v3/roles/199fa373d2ff46b7a0c430b4a8c056e9 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:02:22.766821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd7f0e-dc84-4d15-ba19-1de2b19ab8b0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:22.831317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:22.870503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd7f0e-dc84-4d15-ba19-1de2b19ab8b0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:22.902987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c34c7-0e2d-44d7-85c7-d9e92127095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:22.903603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c34c7-0e2d-44d7-85c7-d9e92127095a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:22.903874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c34c7-0e2d-44d7-85c7-d9e92127095a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:22.904139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c34c7-0e2d-44d7-85c7-d9e92127095a None admin] PATH_INFO: `/v3/users/be71e5d7713c42eeafa9734d86f5149c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:22.905262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c34c7-0e2d-44d7-85c7-d9e92127095a None admin] RBAC: Authorizing `identity:get_user(user_id=be71e5d7713c42eeafa9734d86f5149c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:22.906677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c34c7-0e2d-44d7-85c7-d9e92127095a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:22.908223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 570/1121] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:22 2022] GET /identity/v3/users/be71e5d7713c42eeafa9734d86f5149c => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:22.914165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971604f-64c8-4425-8aa4-c026f9419313 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:23.013860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971604f-64c8-4425-8aa4-c026f9419313 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:23.046446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2af20-eb9b-43a7-a312-f492c8306503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:23.047058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2af20-eb9b-43a7-a312-f492c8306503 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:23.047333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2af20-eb9b-43a7-a312-f492c8306503 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:23.047602 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2af20-eb9b-43a7-a312-f492c8306503 None admin] PATH_INFO: `/v3/projects/20c05749a352455a98813755480ce1d8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:23.048705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2af20-eb9b-43a7-a312-f492c8306503 None admin] RBAC: Authorizing `identity:get_project(project_id=20c05749a352455a98813755480ce1d8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:23.050098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2af20-eb9b-43a7-a312-f492c8306503 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:23.051679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 552/1122] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:22 2022] GET /identity/v3/projects/20c05749a352455a98813755480ce1d8 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:23.058827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c34c7-0e2d-44d7-85c7-d9e92127095a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:23.162528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c34c7-0e2d-44d7-85c7-d9e92127095a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:23.191597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:23.198680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89ea73-a121-4505-962e-013bf81db0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:23.199329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ea73-a121-4505-962e-013bf81db0ac None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:23.199601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ea73-a121-4505-962e-013bf81db0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:23.199865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89ea73-a121-4505-962e-013bf81db0ac None admin] PATH_INFO: `/v3/projects/20c05749a352455a98813755480ce1d8/users/be71e5d7713c42eeafa9734d86f5149c/roles/199fa373d2ff46b7a0c430b4a8c056e9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:23.201991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89ea73-a121-4505-962e-013bf81db0ac None admin] RBAC: Authorizing `identity:create_grant(project_id=20c05749a352455a98813755480ce1d8, user_id=be71e5d7713c42eeafa9734d86f5149c, role_id=199fa373d2ff46b7a0c430b4a8c056e9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:23.203438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89ea73-a121-4505-962e-013bf81db0ac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:23.216258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 571/1123] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:02:23 2022] PUT /identity/v3/projects/20c05749a352455a98813755480ce1d8/users/be71e5d7713c42eeafa9734d86f5149c/roles/199fa373d2ff46b7a0c430b4a8c056e9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:02:25.214488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18467a9-080c-4684-8d5d-5e031222d757 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:25.215017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18467a9-080c-4684-8d5d-5e031222d757 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:25.215017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18467a9-080c-4684-8d5d-5e031222d757 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:25.216341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 553/1124] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:25.221423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c18a5-f626-4961-ac63-84219defe847 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:25.221803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c18a5-f626-4961-ac63-84219defe847 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:25.221964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c18a5-f626-4961-ac63-84219defe847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:25.240242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5e2c18a5-f626-4961-ac63-84219defe847 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:25.241433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5e2c18a5-f626-4961-ac63-84219defe847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:25.307966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 572/1125] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:25.330935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18467a9-080c-4684-8d5d-5e031222d757 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:25.389878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:25.436136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18467a9-080c-4684-8d5d-5e031222d757 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:25.470024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28e81f-7c9f-4d95-b01d-c5e030b7121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:25.470586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28e81f-7c9f-4d95-b01d-c5e030b7121f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:25.470867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28e81f-7c9f-4d95-b01d-c5e030b7121f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:25.471142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28e81f-7c9f-4d95-b01d-c5e030b7121f None admin] PATH_INFO: `/v3/roles/199fa373d2ff46b7a0c430b4a8c056e9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:25.472553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28e81f-7c9f-4d95-b01d-c5e030b7121f None admin] RBAC: Authorizing `identity:get_role(role_id=199fa373d2ff46b7a0c430b4a8c056e9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:25.474017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28e81f-7c9f-4d95-b01d-c5e030b7121f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:25.475331 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 554/1126] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:25 2022] GET /identity/v3/roles/199fa373d2ff46b7a0c430b4a8c056e9 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:25.482613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c18a5-f626-4961-ac63-84219defe847 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:25.584003 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c18a5-f626-4961-ac63-84219defe847 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:25.603998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:25.618921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c1850-b584-4ecc-b7d3-b872ed865080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:25.619539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1850-b584-4ecc-b7d3-b872ed865080 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:25.619814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1850-b584-4ecc-b7d3-b872ed865080 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:25.620085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c1850-b584-4ecc-b7d3-b872ed865080 None admin] PATH_INFO: `/v3/users/be71e5d7713c42eeafa9734d86f5149c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:25.621223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c1850-b584-4ecc-b7d3-b872ed865080 None admin] RBAC: Authorizing `identity:get_user(user_id=be71e5d7713c42eeafa9734d86f5149c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:25.622652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c1850-b584-4ecc-b7d3-b872ed865080 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:25.624222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 573/1127] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:25 2022] GET /identity/v3/users/be71e5d7713c42eeafa9734d86f5149c => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:25.629970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28e81f-7c9f-4d95-b01d-c5e030b7121f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:25.730723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28e81f-7c9f-4d95-b01d-c5e030b7121f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:25.753392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:25.764617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478e38-0bdb-4477-8bb9-4fcdfb227315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:25.765218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478e38-0bdb-4477-8bb9-4fcdfb227315 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:25.765496 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478e38-0bdb-4477-8bb9-4fcdfb227315 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:25.765767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478e38-0bdb-4477-8bb9-4fcdfb227315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:25.841690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c478e38-0bdb-4477-8bb9-4fcdfb227315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:25.843131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c478e38-0bdb-4477-8bb9-4fcdfb227315 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:25.956459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 555/1128] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:25.962513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c1850-b584-4ecc-b7d3-b872ed865080 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:26.057667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c1850-b584-4ecc-b7d3-b872ed865080 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:26.091139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afa084-e606-4f5b-a1e3-5efa49836bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:26.091765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afa084-e606-4f5b-a1e3-5efa49836bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:26.092043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afa084-e606-4f5b-a1e3-5efa49836bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:26.092313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afa084-e606-4f5b-a1e3-5efa49836bf1 None admin] PATH_INFO: `/v3/projects/20c05749a352455a98813755480ce1d8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:26.093446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afa084-e606-4f5b-a1e3-5efa49836bf1 None admin] RBAC: Authorizing `identity:get_project(project_id=20c05749a352455a98813755480ce1d8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:26.094870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afa084-e606-4f5b-a1e3-5efa49836bf1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:26.096539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 574/1129] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:25 2022] GET /identity/v3/projects/20c05749a352455a98813755480ce1d8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:26.102609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478e38-0bdb-4477-8bb9-4fcdfb227315 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:26.127715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:26.202435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478e38-0bdb-4477-8bb9-4fcdfb227315 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:26.234641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432762d3-cc06-46bd-9bc9-e1c42085a635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:26.235243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432762d3-cc06-46bd-9bc9-e1c42085a635 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:26.235488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432762d3-cc06-46bd-9bc9-e1c42085a635 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:26.235753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432762d3-cc06-46bd-9bc9-e1c42085a635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:26.236484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432762d3-cc06-46bd-9bc9-e1c42085a635 None admin] RBAC: Adding query filter params (role.id=199fa373d2ff46b7a0c430b4a8c056e9, scope.project.id=20c05749a352455a98813755480ce1d8, user.id=be71e5d7713c42eeafa9734d86f5149c) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:26.236789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432762d3-cc06-46bd-9bc9-e1c42085a635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:26.238190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432762d3-cc06-46bd-9bc9-e1c42085a635 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:26.244281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 556/1130] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:26 2022] GET /identity/v3/role_assignments?user.id=be71e5d7713c42eeafa9734d86f5149c&scope.project.id=20c05749a352455a98813755480ce1d8&role.id=199fa373d2ff46b7a0c430b4a8c056e9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:28.269017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cf8a7-d29c-448d-ab10-22f537605f40 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:28.269581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cf8a7-d29c-448d-ab10-22f537605f40 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:28.269581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cf8a7-d29c-448d-ab10-22f537605f40 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:28.270875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 575/1131] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:28.276099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28c6a-5c52-4243-bfe6-ed603b19b651 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:28.276529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28c6a-5c52-4243-bfe6-ed603b19b651 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:28.276668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28c6a-5c52-4243-bfe6-ed603b19b651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:28.294290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d4c28c6a-5c52-4243-bfe6-ed603b19b651 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:28.295467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d4c28c6a-5c52-4243-bfe6-ed603b19b651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:28.361023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 557/1132] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:28.385015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cf8a7-d29c-448d-ab10-22f537605f40 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:28.441365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:28.485289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cf8a7-d29c-448d-ab10-22f537605f40 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:28.518677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f935cc7-29c1-40cd-9d44-35be6da11803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:28.519341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f935cc7-29c1-40cd-9d44-35be6da11803 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:28.519568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f935cc7-29c1-40cd-9d44-35be6da11803 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:28.519835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f935cc7-29c1-40cd-9d44-35be6da11803 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:28.520964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f935cc7-29c1-40cd-9d44-35be6da11803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:28.522384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f935cc7-29c1-40cd-9d44-35be6da11803 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:28.524240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 576/1133] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:28.530056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c28c6a-5c52-4243-bfe6-ed603b19b651 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:28.548270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:28.627030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c28c6a-5c52-4243-bfe6-ed603b19b651 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:28.659670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302fd81-750b-4a9c-a9d8-109439086dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:28.660264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302fd81-750b-4a9c-a9d8-109439086dea None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:28.660572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302fd81-750b-4a9c-a9d8-109439086dea None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:28.660843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302fd81-750b-4a9c-a9d8-109439086dea None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:28.661484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302fd81-750b-4a9c-a9d8-109439086dea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:28.662901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302fd81-750b-4a9c-a9d8-109439086dea None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:28.689745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 558/1134] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:02:28 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:02:30.711783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b6c5-c106-4ae4-b755-758c2fa87a73 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:30.712300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b6c5-c106-4ae4-b755-758c2fa87a73 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:30.712300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764b6c5-c106-4ae4-b755-758c2fa87a73 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:30.713664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 577/1135] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:30.718933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914ccbf-e0d7-4468-bbdc-c8dd41f45791 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:30.719370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914ccbf-e0d7-4468-bbdc-c8dd41f45791 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:30.719486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914ccbf-e0d7-4468-bbdc-c8dd41f45791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:30.737813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b914ccbf-e0d7-4468-bbdc-c8dd41f45791 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:30.738992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b914ccbf-e0d7-4468-bbdc-c8dd41f45791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:30.798922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:30.804093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 559/1136] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:30.828887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764b6c5-c106-4ae4-b755-758c2fa87a73 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:30.927710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764b6c5-c106-4ae4-b755-758c2fa87a73 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:30.951699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:30.962421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d9c80-619f-464c-b0f2-ea9a965cb2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:30.962978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d9c80-619f-464c-b0f2-ea9a965cb2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:30.963257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d9c80-619f-464c-b0f2-ea9a965cb2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:30.963530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d9c80-619f-464c-b0f2-ea9a965cb2d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:30.964656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d9c80-619f-464c-b0f2-ea9a965cb2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:30.966185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d9c80-619f-464c-b0f2-ea9a965cb2d9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:30.967732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 578/1137] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:02:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:30.973407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914ccbf-e0d7-4468-bbdc-c8dd41f45791 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:31.069492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914ccbf-e0d7-4468-bbdc-c8dd41f45791 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:31.102946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe78ae2-d22f-4a74-8121-fb9ce0c2a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:31.103536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe78ae2-d22f-4a74-8121-fb9ce0c2a75f None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:31.103809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe78ae2-d22f-4a74-8121-fb9ce0c2a75f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:31.104076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe78ae2-d22f-4a74-8121-fb9ce0c2a75f None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:31.104742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe78ae2-d22f-4a74-8121-fb9ce0c2a75f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:31.106150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe78ae2-d22f-4a74-8121-fb9ce0c2a75f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:31.127481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 560/1138] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:02:30 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:02:33.204301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7230-7934-488e-93f3-10a4f2cfba6c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:33.204901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7230-7934-488e-93f3-10a4f2cfba6c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:33.204901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7230-7934-488e-93f3-10a4f2cfba6c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:33.206198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 579/1139] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:33.211808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caca4-9478-49c1-b631-03e48f693330 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:33.212304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caca4-9478-49c1-b631-03e48f693330 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:33.212414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caca4-9478-49c1-b631-03e48f693330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:33.234882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7a0caca4-9478-49c1-b631-03e48f693330 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:33.236078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7a0caca4-9478-49c1-b631-03e48f693330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:33.299241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:33.318207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 561/1140] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:33.344852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a7230-7934-488e-93f3-10a4f2cfba6c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:33.456213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a7230-7934-488e-93f3-10a4f2cfba6c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:33.495906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0f379-d6ae-415e-a89d-b089b83584a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:33.496782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0f379-d6ae-415e-a89d-b089b83584a9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:33.497790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0f379-d6ae-415e-a89d-b089b83584a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:33.497790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0f379-d6ae-415e-a89d-b089b83584a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:33.511659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0f379-d6ae-415e-a89d-b089b83584a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-27a0f379-d6ae-415e-a89d-b089b83584a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:33.516592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:33.517935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:33.519135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 580/1141] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:33.527334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0caca4-9478-49c1-b631-03e48f693330 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:33.649986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0caca4-9478-49c1-b631-03e48f693330 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:33.691575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:33.692609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:33.692916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:33.693193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:33.693864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:33.694170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:33.695745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:33.706875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 562/1142] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:33.713823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0f379-d6ae-415e-a89d-b089b83584a9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:33.778452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:33.835722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0f379-d6ae-415e-a89d-b089b83584a9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:33.874725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68a539-d471-46af-9557-1ff5be8ea751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:33.875863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68a539-d471-46af-9557-1ff5be8ea751 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:33.876330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68a539-d471-46af-9557-1ff5be8ea751 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:33.876827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68a539-d471-46af-9557-1ff5be8ea751 None admin] PATH_INFO: `/v3/users/aa979a15c59a44b69ec251910e4da8d7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:33.892563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68a539-d471-46af-9557-1ff5be8ea751 None admin] RBAC: Authorizing `identity:get_user(user_id=aa979a15c59a44b69ec251910e4da8d7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:33.894021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68a539-d471-46af-9557-1ff5be8ea751 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:33.895669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 581/1143] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:33 2022] GET /identity/v3/users/aa979a15c59a44b69ec251910e4da8d7 => generated 286 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:33.905181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:33.955821 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:34.006637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8c78-290b-4386-82b1-defa70ded34b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:34.041797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933c46-6297-4195-9e54-ce6fa20890d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:34.042397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933c46-6297-4195-9e54-ce6fa20890d3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:34.042671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933c46-6297-4195-9e54-ce6fa20890d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:34.042938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933c46-6297-4195-9e54-ce6fa20890d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:34.109389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3933c46-6297-4195-9e54-ce6fa20890d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:34.110819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3933c46-6297-4195-9e54-ce6fa20890d3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:34.157348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:34.208700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 563/1144] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:34.215133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68a539-d471-46af-9557-1ff5be8ea751 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:34.311281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68a539-d471-46af-9557-1ff5be8ea751 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:34.322393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:34.345899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcda9e7-0040-44b5-823f-9be34d09aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:34.346512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcda9e7-0040-44b5-823f-9be34d09aa90 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:34.346783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcda9e7-0040-44b5-823f-9be34d09aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:34.347050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcda9e7-0040-44b5-823f-9be34d09aa90 None admin] PATH_INFO: `/v3/projects/1acd2ac315c04500b5074880042bfaf2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:34.348133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcda9e7-0040-44b5-823f-9be34d09aa90 None admin] RBAC: Authorizing `identity:get_project(project_id=1acd2ac315c04500b5074880042bfaf2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:34.349578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcda9e7-0040-44b5-823f-9be34d09aa90 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:34.351170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 582/1145] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:34 2022] GET /identity/v3/projects/1acd2ac315c04500b5074880042bfaf2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:34.358040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933c46-6297-4195-9e54-ce6fa20890d3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:34.456222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933c46-6297-4195-9e54-ce6fa20890d3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:34.490454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4cb51-85f8-46bc-a586-cb5571a493d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:34.491049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cb51-85f8-46bc-a586-cb5571a493d5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:34.491327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cb51-85f8-46bc-a586-cb5571a493d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:34.491594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4cb51-85f8-46bc-a586-cb5571a493d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:34.492375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4cb51-85f8-46bc-a586-cb5571a493d5 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=1acd2ac315c04500b5074880042bfaf2, user.id=aa979a15c59a44b69ec251910e4da8d7) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:34.492707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4cb51-85f8-46bc-a586-cb5571a493d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:34.494112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4cb51-85f8-46bc-a586-cb5571a493d5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:34.499548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 564/1146] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:34 2022] GET /identity/v3/role_assignments?user.id=aa979a15c59a44b69ec251910e4da8d7&scope.project.id=1acd2ac315c04500b5074880042bfaf2&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:36.461836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada8fcc-bf7d-465e-94b6-ee24a9c0702e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:36.462346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada8fcc-bf7d-465e-94b6-ee24a9c0702e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:36.462346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada8fcc-bf7d-465e-94b6-ee24a9c0702e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:36.463879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 583/1147] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:36.468951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219600f0-ce82-48cf-9cb3-1601743e112c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:36.469377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219600f0-ce82-48cf-9cb3-1601743e112c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:36.469494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219600f0-ce82-48cf-9cb3-1601743e112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:36.486920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-219600f0-ce82-48cf-9cb3-1601743e112c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:36.488085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-219600f0-ce82-48cf-9cb3-1601743e112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:36.495919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:36.553638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 565/1148] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:36.576847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada8fcc-bf7d-465e-94b6-ee24a9c0702e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:36.681523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada8fcc-bf7d-465e-94b6-ee24a9c0702e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:36.714250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49813c9-5a3d-4081-a320-c56a51b5956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:36.714865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49813c9-5a3d-4081-a320-c56a51b5956a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:36.715139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49813c9-5a3d-4081-a320-c56a51b5956a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:36.715406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49813c9-5a3d-4081-a320-c56a51b5956a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:36.725025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49813c9-5a3d-4081-a320-c56a51b5956a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e49813c9-5a3d-4081-a320-c56a51b5956a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:36.729926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:36.731059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:36.732113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 584/1149] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:36.737115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219600f0-ce82-48cf-9cb3-1601743e112c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:36.835145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219600f0-ce82-48cf-9cb3-1601743e112c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:36.869404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:36.870004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:36.870282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:36.870556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:36.871210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:36.871510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:36.872890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:36.880611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 566/1150] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:36.886729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49813c9-5a3d-4081-a320-c56a51b5956a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:36.927013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:36.989904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49813c9-5a3d-4081-a320-c56a51b5956a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:37.022339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2105dc9-db73-4886-aa24-7cc11f7bdad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:37.022957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2105dc9-db73-4886-aa24-7cc11f7bdad6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:37.023253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2105dc9-db73-4886-aa24-7cc11f7bdad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:37.023524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2105dc9-db73-4886-aa24-7cc11f7bdad6 None admin] PATH_INFO: `/v3/users/aa979a15c59a44b69ec251910e4da8d7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:37.024782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2105dc9-db73-4886-aa24-7cc11f7bdad6 None admin] RBAC: Authorizing `identity:get_user(user_id=aa979a15c59a44b69ec251910e4da8d7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:37.026209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2105dc9-db73-4886-aa24-7cc11f7bdad6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:37.027769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 585/1151] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:36 2022] GET /identity/v3/users/aa979a15c59a44b69ec251910e4da8d7 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:37.033632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:37.037331 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:37.132351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834ccf9-d0c6-4647-9a79-5d191c45381f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:37.164649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b464c25-396e-448a-b059-6104db769076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:37.165244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b464c25-396e-448a-b059-6104db769076 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:37.165517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b464c25-396e-448a-b059-6104db769076 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:37.165785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b464c25-396e-448a-b059-6104db769076 None admin] PATH_INFO: `/v3/projects/1acd2ac315c04500b5074880042bfaf2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:37.166876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b464c25-396e-448a-b059-6104db769076 None admin] RBAC: Authorizing `identity:get_project(project_id=1acd2ac315c04500b5074880042bfaf2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:37.168271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b464c25-396e-448a-b059-6104db769076 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:37.169881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 567/1152] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:37 2022] GET /identity/v3/projects/1acd2ac315c04500b5074880042bfaf2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:37.175806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2105dc9-db73-4886-aa24-7cc11f7bdad6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:37.270945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2105dc9-db73-4886-aa24-7cc11f7bdad6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:37.278523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:37.306670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c9c8d-c931-4cc4-99e4-1567b136fce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:37.307313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9c8d-c931-4cc4-99e4-1567b136fce9 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:37.307584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9c8d-c931-4cc4-99e4-1567b136fce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:37.307851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9c8d-c931-4cc4-99e4-1567b136fce9 None admin] PATH_INFO: `/v3/projects/1acd2ac315c04500b5074880042bfaf2/users/aa979a15c59a44b69ec251910e4da8d7/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:37.310060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c9c8d-c931-4cc4-99e4-1567b136fce9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1acd2ac315c04500b5074880042bfaf2, user_id=aa979a15c59a44b69ec251910e4da8d7, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:37.311541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c9c8d-c931-4cc4-99e4-1567b136fce9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:37.324906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 586/1153] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:02:37 2022] PUT /identity/v3/projects/1acd2ac315c04500b5074880042bfaf2/users/aa979a15c59a44b69ec251910e4da8d7/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:02:39.368604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef13d-bb33-41f2-9ea5-42bfa7d93166 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:39.369128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef13d-bb33-41f2-9ea5-42bfa7d93166 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:39.369128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ef13d-bb33-41f2-9ea5-42bfa7d93166 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:39.370459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 568/1154] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:39.375899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311dd62-b45c-4ce3-b105-a10ef02b46db None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:39.376593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311dd62-b45c-4ce3-b105-a10ef02b46db None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:39.376825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311dd62-b45c-4ce3-b105-a10ef02b46db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:39.395932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4311dd62-b45c-4ce3-b105-a10ef02b46db None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:39.397166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4311dd62-b45c-4ce3-b105-a10ef02b46db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:39.462734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 587/1155] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:39.486029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ef13d-bb33-41f2-9ea5-42bfa7d93166 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:39.557525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:39.588379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ef13d-bb33-41f2-9ea5-42bfa7d93166 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:39.621487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831fa46-f810-4814-9255-c057e833f4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:39.622040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831fa46-f810-4814-9255-c057e833f4be None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:39.622312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831fa46-f810-4814-9255-c057e833f4be None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:39.622577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831fa46-f810-4814-9255-c057e833f4be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:39.632318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831fa46-f810-4814-9255-c057e833f4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-6831fa46-f810-4814-9255-c057e833f4be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:39.637735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:39.638844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:39.639891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 569/1156] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:39.644930 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311dd62-b45c-4ce3-b105-a10ef02b46db None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:39.737193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:39.744498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311dd62-b45c-4ce3-b105-a10ef02b46db None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:39.779144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:39.779748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:39.780027 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:39.780299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:39.780987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:39.781293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:39.782655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:39.791196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 588/1157] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:39.796972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831fa46-f810-4814-9255-c057e833f4be None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:39.895656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831fa46-f810-4814-9255-c057e833f4be None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:39.924254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:39.929782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c308b8-c29c-41b0-a78a-9a744eb44f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:39.930403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c308b8-c29c-41b0-a78a-9a744eb44f9f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:39.930683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c308b8-c29c-41b0-a78a-9a744eb44f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:39.930956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c308b8-c29c-41b0-a78a-9a744eb44f9f None admin] PATH_INFO: `/v3/users/aa979a15c59a44b69ec251910e4da8d7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:39.932054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c308b8-c29c-41b0-a78a-9a744eb44f9f None admin] RBAC: Authorizing `identity:get_user(user_id=aa979a15c59a44b69ec251910e4da8d7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:39.933504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c308b8-c29c-41b0-a78a-9a744eb44f9f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:39.935441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 570/1158] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:39 2022] GET /identity/v3/users/aa979a15c59a44b69ec251910e4da8d7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:39.941303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:40.040380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a23b9c-0b3a-439f-aaf8-86e8232683bf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:40.073624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3236b9-d713-416e-98eb-d8286bb5097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:40.074519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3236b9-d713-416e-98eb-d8286bb5097b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:40.074806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3236b9-d713-416e-98eb-d8286bb5097b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:40.075146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3236b9-d713-416e-98eb-d8286bb5097b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:40.101688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:40.141512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3236b9-d713-416e-98eb-d8286bb5097b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:40.142947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3236b9-d713-416e-98eb-d8286bb5097b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:40.236105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 589/1159] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:40.242698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c308b8-c29c-41b0-a78a-9a744eb44f9f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:40.338830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c308b8-c29c-41b0-a78a-9a744eb44f9f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:40.372810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c0a0-209b-4a15-b7b0-0f2d4ee274b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:40.373432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c0a0-209b-4a15-b7b0-0f2d4ee274b1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:40.373742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c0a0-209b-4a15-b7b0-0f2d4ee274b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:40.373982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c0a0-209b-4a15-b7b0-0f2d4ee274b1 None admin] PATH_INFO: `/v3/projects/1acd2ac315c04500b5074880042bfaf2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:40.375092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7c0a0-209b-4a15-b7b0-0f2d4ee274b1 None admin] RBAC: Authorizing `identity:get_project(project_id=1acd2ac315c04500b5074880042bfaf2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:40.376537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7c0a0-209b-4a15-b7b0-0f2d4ee274b1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:40.378136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 571/1160] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:40 2022] GET /identity/v3/projects/1acd2ac315c04500b5074880042bfaf2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:40.385383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3236b9-d713-416e-98eb-d8286bb5097b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:40.448713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:40.501311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3236b9-d713-416e-98eb-d8286bb5097b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:40.535433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efabd14-6683-4e8b-ab7a-6c83e235fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:40.536028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efabd14-6683-4e8b-ab7a-6c83e235fefd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:40.536308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efabd14-6683-4e8b-ab7a-6c83e235fefd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:40.536612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efabd14-6683-4e8b-ab7a-6c83e235fefd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:40.537331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efabd14-6683-4e8b-ab7a-6c83e235fefd None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=1acd2ac315c04500b5074880042bfaf2, user.id=aa979a15c59a44b69ec251910e4da8d7) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:40.537637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efabd14-6683-4e8b-ab7a-6c83e235fefd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:40.539352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efabd14-6683-4e8b-ab7a-6c83e235fefd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:40.544912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 590/1161] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:40 2022] GET /identity/v3/role_assignments?user.id=aa979a15c59a44b69ec251910e4da8d7&scope.project.id=1acd2ac315c04500b5074880042bfaf2&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:42.586518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890228f-050c-4477-ba4c-88d12d84849f None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:42.587065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890228f-050c-4477-ba4c-88d12d84849f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:42.587065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890228f-050c-4477-ba4c-88d12d84849f None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:42.588371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 572/1162] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:42.593434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2900e-3e9c-4d95-9df7-cdd2fed02ed7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:42.593876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2900e-3e9c-4d95-9df7-cdd2fed02ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:42.593999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2900e-3e9c-4d95-9df7-cdd2fed02ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:42.612099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d8e2900e-3e9c-4d95-9df7-cdd2fed02ed7 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:42.613297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d8e2900e-3e9c-4d95-9df7-cdd2fed02ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:42.676762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 591/1163] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:42.698915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890228f-050c-4477-ba4c-88d12d84849f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:42.767468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:42.802938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890228f-050c-4477-ba4c-88d12d84849f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:42.837389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357fdf3-5a5c-481d-88af-bc76cf171914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:42.837986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357fdf3-5a5c-481d-88af-bc76cf171914 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:42.838266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357fdf3-5a5c-481d-88af-bc76cf171914 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:42.838542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357fdf3-5a5c-481d-88af-bc76cf171914 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:42.903858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357fdf3-5a5c-481d-88af-bc76cf171914 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:42.905325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357fdf3-5a5c-481d-88af-bc76cf171914 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:42.967734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:43.001335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 573/1164] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:43.007216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2900e-3e9c-4d95-9df7-cdd2fed02ed7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:43.057478 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:43.105939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2900e-3e9c-4d95-9df7-cdd2fed02ed7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:43.138945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15bb10-35ff-42e8-afe0-541485c598bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:43.139554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15bb10-35ff-42e8-afe0-541485c598bf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:43.139827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15bb10-35ff-42e8-afe0-541485c598bf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:43.140093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15bb10-35ff-42e8-afe0-541485c598bf None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:43.146370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15bb10-35ff-42e8-afe0-541485c598bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:43.147791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15bb10-35ff-42e8-afe0-541485c598bf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9d15bb10-35ff-42e8-afe0-541485c598bf None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:43.159231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:43.160403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:43.161287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 592/1165] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:02:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:43.167398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357fdf3-5a5c-481d-88af-bc76cf171914 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:43.264086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357fdf3-5a5c-481d-88af-bc76cf171914 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:43.297966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec93ed6-d223-4c01-b4f9-d06b2cbc48ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:43.298560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec93ed6-d223-4c01-b4f9-d06b2cbc48ab None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:43.298837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec93ed6-d223-4c01-b4f9-d06b2cbc48ab None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:43.299110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec93ed6-d223-4c01-b4f9-d06b2cbc48ab None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:43.299745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec93ed6-d223-4c01-b4f9-d06b2cbc48ab None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:43.300048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec93ed6-d223-4c01-b4f9-d06b2cbc48ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:43.301430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec93ed6-d223-4c01-b4f9-d06b2cbc48ab None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:43.308681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 574/1166] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Aug 9 19:02:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:45.324963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ca98-0e0c-4703-870d-2c70f2a36352 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:45.325490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ca98-0e0c-4703-870d-2c70f2a36352 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:45.325490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355ca98-0e0c-4703-870d-2c70f2a36352 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:45.327018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 593/1167] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:45.332407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4647069-d2f7-4827-97de-78aac676394d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:45.332800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4647069-d2f7-4827-97de-78aac676394d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:45.332991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4647069-d2f7-4827-97de-78aac676394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:45.350931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d4647069-d2f7-4827-97de-78aac676394d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:45.352919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d4647069-d2f7-4827-97de-78aac676394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:45.378395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:45.420775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 575/1168] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:45.443970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355ca98-0e0c-4703-870d-2c70f2a36352 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:45.546434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355ca98-0e0c-4703-870d-2c70f2a36352 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:45.558995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:45.580370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68709499-4ab3-4127-b425-06180dc38115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:45.580931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68709499-4ab3-4127-b425-06180dc38115 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:45.581210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68709499-4ab3-4127-b425-06180dc38115 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:45.581484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68709499-4ab3-4127-b425-06180dc38115 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:45.582147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68709499-4ab3-4127-b425-06180dc38115 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:45.583512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68709499-4ab3-4127-b425-06180dc38115 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:45.599840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 594/1169] 10.222.0.70 () {60 vars in 1138 bytes} [Tue Aug 9 19:02:45 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:02:47.660946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95147ca6-f064-4304-ad53-66bdcba83b31 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:47.661450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95147ca6-f064-4304-ad53-66bdcba83b31 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:47.661563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95147ca6-f064-4304-ad53-66bdcba83b31 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:47.662814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 576/1170] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:47.668883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d002e6-0d2c-408c-b392-9471aeed3718 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:47.669302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d002e6-0d2c-408c-b392-9471aeed3718 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:47.669434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d002e6-0d2c-408c-b392-9471aeed3718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:47.687165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-13d002e6-0d2c-408c-b392-9471aeed3718 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:47.688333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-13d002e6-0d2c-408c-b392-9471aeed3718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:47.753648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 595/1171] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:47.777137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95147ca6-f064-4304-ad53-66bdcba83b31 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:47.876689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95147ca6-f064-4304-ad53-66bdcba83b31 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:47.909409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41381a39-6b0a-4c37-bdce-df8b9357d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:47.910021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41381a39-6b0a-4c37-bdce-df8b9357d7cf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:47.910302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41381a39-6b0a-4c37-bdce-df8b9357d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:47.910574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41381a39-6b0a-4c37-bdce-df8b9357d7cf None admin] PATH_INFO: `/v3/domains/bc26ad98bfd04617953c40611d469898` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:47.921208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41381a39-6b0a-4c37-bdce-df8b9357d7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=bc26ad98bfd04617953c40611d469898)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:47.922572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41381a39-6b0a-4c37-bdce-df8b9357d7cf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:47.924114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 577/1172] 10.222.0.70 () {56 vars in 1183 bytes} [Tue Aug 9 19:02:47 2022] GET /identity/v3/domains/bc26ad98bfd04617953c40611d469898 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:47.930188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d002e6-0d2c-408c-b392-9471aeed3718 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:48.026076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d002e6-0d2c-408c-b392-9471aeed3718 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:48.027564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:48.060260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c57c09-c57e-498b-8113-ac28af156e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:48.060883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c57c09-c57e-498b-8113-ac28af156e7e None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:48.061159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c57c09-c57e-498b-8113-ac28af156e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:48.061430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c57c09-c57e-498b-8113-ac28af156e7e None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:48.062058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c57c09-c57e-498b-8113-ac28af156e7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:48.063486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c57c09-c57e-498b-8113-ac28af156e7e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:48.089845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 596/1173] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:02:47 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:02:50.116032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5ca86-a531-4747-ac25-30769378d28d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:50.117129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5ca86-a531-4747-ac25-30769378d28d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:50.117129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5ca86-a531-4747-ac25-30769378d28d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:50.118622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 578/1174] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:50.123698 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e2e65-5962-4554-a2c4-6bfc439be620 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:50.124095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e2e65-5962-4554-a2c4-6bfc439be620 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:50.124242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e2e65-5962-4554-a2c4-6bfc439be620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:50.142632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c12e2e65-5962-4554-a2c4-6bfc439be620 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:50.143815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c12e2e65-5962-4554-a2c4-6bfc439be620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:50.209035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 597/1175] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:50.233950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5ca86-a531-4747-ac25-30769378d28d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:50.259450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:50.336283 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5ca86-a531-4747-ac25-30769378d28d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:50.369715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9247913-7fd3-4ea0-88cd-07b84bb87788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:50.370266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247913-7fd3-4ea0-88cd-07b84bb87788 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:50.370541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247913-7fd3-4ea0-88cd-07b84bb87788 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:50.370807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9247913-7fd3-4ea0-88cd-07b84bb87788 None admin] PATH_INFO: `/v3/domains/bc26ad98bfd04617953c40611d469898` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:50.371900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9247913-7fd3-4ea0-88cd-07b84bb87788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bc26ad98bfd04617953c40611d469898)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:50.373458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9247913-7fd3-4ea0-88cd-07b84bb87788 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:50.374989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 579/1176] 10.222.0.70 () {56 vars in 1183 bytes} [Tue Aug 9 19:02:50 2022] GET /identity/v3/domains/bc26ad98bfd04617953c40611d469898 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:50.381053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e2e65-5962-4554-a2c4-6bfc439be620 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:50.465948 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:50.482748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e2e65-5962-4554-a2c4-6bfc439be620 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:50.515449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef681153-0466-4000-9932-c56aab5977e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:50.516044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef681153-0466-4000-9932-c56aab5977e8 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:50.516318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef681153-0466-4000-9932-c56aab5977e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:50.516624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef681153-0466-4000-9932-c56aab5977e8 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:50.517269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef681153-0466-4000-9932-c56aab5977e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:50.518688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef681153-0466-4000-9932-c56aab5977e8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:50.541444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 598/1177] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:02:50 2022] POST /identity/v3/users => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:02:52.602553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb73fe0-5ecc-4a46-be54-edef5cf09516 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:52.603143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb73fe0-5ecc-4a46-be54-edef5cf09516 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:52.603143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb73fe0-5ecc-4a46-be54-edef5cf09516 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:52.604415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 580/1178] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:52.609588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf017d2-4a5f-40cc-a656-dfb631e47b7a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:52.610015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf017d2-4a5f-40cc-a656-dfb631e47b7a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:52.610124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf017d2-4a5f-40cc-a656-dfb631e47b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:52.628035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9bf017d2-4a5f-40cc-a656-dfb631e47b7a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:52.629253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9bf017d2-4a5f-40cc-a656-dfb631e47b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:52.693190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 599/1179] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:52.717134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb73fe0-5ecc-4a46-be54-edef5cf09516 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:52.811285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:52.822837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb73fe0-5ecc-4a46-be54-edef5cf09516 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:52.856787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02ddc-8c90-49e4-8df0-dac23edfa5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:52.857413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02ddc-8c90-49e4-8df0-dac23edfa5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:52.857691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02ddc-8c90-49e4-8df0-dac23edfa5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:52.857963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02ddc-8c90-49e4-8df0-dac23edfa5c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:52.868195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02ddc-8c90-49e4-8df0-dac23edfa5c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4da02ddc-8c90-49e4-8df0-dac23edfa5c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:52.872803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:52.873947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:52.875195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 581/1180] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:52.880132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf017d2-4a5f-40cc-a656-dfb631e47b7a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:52.947997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:52.978387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf017d2-4a5f-40cc-a656-dfb631e47b7a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:53.010843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:53.011434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:53.011707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:53.012005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:53.012658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:53.012956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:53.014308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:53.021928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 600/1181] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:53.028429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02ddc-8c90-49e4-8df0-dac23edfa5c7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:53.127316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02ddc-8c90-49e4-8df0-dac23edfa5c7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:53.160360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4e509-a181-4ac5-a486-f82f2559c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:53.161007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4e509-a181-4ac5-a486-f82f2559c5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:53.161288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4e509-a181-4ac5-a486-f82f2559c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:53.161563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4e509-a181-4ac5-a486-f82f2559c5d6 None admin] PATH_INFO: `/v3/users/d39b2fccd63c4e3bb53561fd9dceb5ae` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:53.174215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4e509-a181-4ac5-a486-f82f2559c5d6 None admin] RBAC: Authorizing `identity:get_user(user_id=d39b2fccd63c4e3bb53561fd9dceb5ae)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:53.175637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4e509-a181-4ac5-a486-f82f2559c5d6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:53.177230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 582/1182] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:53 2022] GET /identity/v3/users/d39b2fccd63c4e3bb53561fd9dceb5ae => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:53.183913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:53.282215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e18c80-9518-4294-8075-1172fd7164d1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:53.310705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:53.315953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8dbbf-3ec2-43e6-8b01-65fbd5d388db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:53.316569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8dbbf-3ec2-43e6-8b01-65fbd5d388db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:53.316845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8dbbf-3ec2-43e6-8b01-65fbd5d388db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:53.317113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8dbbf-3ec2-43e6-8b01-65fbd5d388db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:53.380700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8dbbf-3ec2-43e6-8b01-65fbd5d388db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:53.382189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8dbbf-3ec2-43e6-8b01-65fbd5d388db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:53.477315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 601/1183] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:53.483216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4e509-a181-4ac5-a486-f82f2559c5d6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:53.494615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:53.583426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4e509-a181-4ac5-a486-f82f2559c5d6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:53.617496 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93bc2d-37b9-4e5d-9c51-ecbe243b3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:53.618158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93bc2d-37b9-4e5d-9c51-ecbe243b3b92 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:53.618375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93bc2d-37b9-4e5d-9c51-ecbe243b3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:53.618643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93bc2d-37b9-4e5d-9c51-ecbe243b3b92 None admin] PATH_INFO: `/v3/projects/792a27137f804243866f534f2df2ed42` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:53.619756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93bc2d-37b9-4e5d-9c51-ecbe243b3b92 None admin] RBAC: Authorizing `identity:get_project(project_id=792a27137f804243866f534f2df2ed42)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:53.621207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93bc2d-37b9-4e5d-9c51-ecbe243b3b92 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:53.622803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 583/1184] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:53 2022] GET /identity/v3/projects/792a27137f804243866f534f2df2ed42 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:53.628880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8dbbf-3ec2-43e6-8b01-65fbd5d388db None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:53.659497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:53.726841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8dbbf-3ec2-43e6-8b01-65fbd5d388db None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:53.759099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42192b1f-ba79-4380-86b0-8d842e0fa467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:53.759688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192b1f-ba79-4380-86b0-8d842e0fa467 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:53.759959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192b1f-ba79-4380-86b0-8d842e0fa467 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:53.760224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192b1f-ba79-4380-86b0-8d842e0fa467 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:53.760966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42192b1f-ba79-4380-86b0-8d842e0fa467 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=792a27137f804243866f534f2df2ed42, user.id=d39b2fccd63c4e3bb53561fd9dceb5ae) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:53.761265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42192b1f-ba79-4380-86b0-8d842e0fa467 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:53.762692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42192b1f-ba79-4380-86b0-8d842e0fa467 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:53.767772 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 602/1185] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:53 2022] GET /identity/v3/role_assignments?user.id=d39b2fccd63c4e3bb53561fd9dceb5ae&scope.project.id=792a27137f804243866f534f2df2ed42&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:55.763455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c59a-01ea-422b-ba32-747ac5e49b19 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:55.764009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c59a-01ea-422b-ba32-747ac5e49b19 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:55.764009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c59a-01ea-422b-ba32-747ac5e49b19 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:55.765343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 584/1186] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:02:55.770451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47404-fdb4-4001-b7c9-659496320228 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:55.770866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47404-fdb4-4001-b7c9-659496320228 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:55.770995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47404-fdb4-4001-b7c9-659496320228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:55.789138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2ee47404-fdb4-4001-b7c9-659496320228 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:55.790335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2ee47404-fdb4-4001-b7c9-659496320228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:55.866220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 603/1187] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:02:55.889154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c59a-01ea-422b-ba32-747ac5e49b19 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:55.967164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:55.990880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c59a-01ea-422b-ba32-747ac5e49b19 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:56.024708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953ffe2-26b7-4bf1-8cee-897ba7636390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:56.025253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953ffe2-26b7-4bf1-8cee-897ba7636390 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:56.025525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953ffe2-26b7-4bf1-8cee-897ba7636390 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:56.025789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953ffe2-26b7-4bf1-8cee-897ba7636390 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:56.035393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953ffe2-26b7-4bf1-8cee-897ba7636390 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-9953ffe2-26b7-4bf1-8cee-897ba7636390 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:56.040761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:56.041847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:02:56.042697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 585/1188] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:56.047888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee47404-fdb4-4001-b7c9-659496320228 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:56.073713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:56.153071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee47404-fdb4-4001-b7c9-659496320228 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:56.197905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:56.198614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:56.199030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:56.199434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:56.200462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:56.200923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:56.203117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:56.214697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 604/1189] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:56.220516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953ffe2-26b7-4bf1-8cee-897ba7636390 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:56.319310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953ffe2-26b7-4bf1-8cee-897ba7636390 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:56.352089 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e08590-c29c-4be7-b91d-a685d9fac5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:56.352727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e08590-c29c-4be7-b91d-a685d9fac5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:56.353004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e08590-c29c-4be7-b91d-a685d9fac5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:56.353274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e08590-c29c-4be7-b91d-a685d9fac5a4 None admin] PATH_INFO: `/v3/users/d39b2fccd63c4e3bb53561fd9dceb5ae` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:56.354374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e08590-c29c-4be7-b91d-a685d9fac5a4 None admin] RBAC: Authorizing `identity:get_user(user_id=d39b2fccd63c4e3bb53561fd9dceb5ae)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:56.355785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e08590-c29c-4be7-b91d-a685d9fac5a4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:56.358421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 586/1190] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:56 2022] GET /identity/v3/users/d39b2fccd63c4e3bb53561fd9dceb5ae => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:56.363384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:56.449689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:56.461991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b5530-1ef8-4a0c-ade3-b2c9339620b7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:56.495234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9d88a-a934-4b23-b9ae-f1f7ce957422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:56.495880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9d88a-a934-4b23-b9ae-f1f7ce957422 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:56.496157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9d88a-a934-4b23-b9ae-f1f7ce957422 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:56.496430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9d88a-a934-4b23-b9ae-f1f7ce957422 None admin] PATH_INFO: `/v3/projects/792a27137f804243866f534f2df2ed42` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:56.497601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9d88a-a934-4b23-b9ae-f1f7ce957422 None admin] RBAC: Authorizing `identity:get_project(project_id=792a27137f804243866f534f2df2ed42)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:56.499030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9d88a-a934-4b23-b9ae-f1f7ce957422 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:56.500780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 605/1191] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:56 2022] GET /identity/v3/projects/792a27137f804243866f534f2df2ed42 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:56.507492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e08590-c29c-4be7-b91d-a685d9fac5a4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:56.513735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:56.617240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e08590-c29c-4be7-b91d-a685d9fac5a4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:56.649593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdda4d-baa4-4914-b569-169650b23bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:56.650233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdda4d-baa4-4914-b569-169650b23bcf None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:56.650514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdda4d-baa4-4914-b569-169650b23bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:56.650785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdda4d-baa4-4914-b569-169650b23bcf None admin] PATH_INFO: `/v3/projects/792a27137f804243866f534f2df2ed42/users/d39b2fccd63c4e3bb53561fd9dceb5ae/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:56.652688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdda4d-baa4-4914-b569-169650b23bcf None admin] RBAC: Authorizing `identity:create_grant(project_id=792a27137f804243866f534f2df2ed42, user_id=d39b2fccd63c4e3bb53561fd9dceb5ae, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:56.654107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdda4d-baa4-4914-b569-169650b23bcf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:56.667264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 587/1192] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:02:56 2022] PUT /identity/v3/projects/792a27137f804243866f534f2df2ed42/users/d39b2fccd63c4e3bb53561fd9dceb5ae/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:02:58.731511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bdbe7-a4d6-44ea-b8e5-c0fad286b045 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:58.732078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bdbe7-a4d6-44ea-b8e5-c0fad286b045 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:58.732078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bdbe7-a4d6-44ea-b8e5-c0fad286b045 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:58.733428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 606/1193] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:02:58.738604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cf07f-11d7-4369-bbfb-f58fbd612359 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:58.739010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cf07f-11d7-4369-bbfb-f58fbd612359 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:58.739156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cf07f-11d7-4369-bbfb-f58fbd612359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:58.757668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-092cf07f-11d7-4369-bbfb-f58fbd612359 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:02:58.758838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-092cf07f-11d7-4369-bbfb-f58fbd612359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:02:58.816691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:58.827050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 588/1194] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:02:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:02:58.851070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bdbe7-a4d6-44ea-b8e5-c0fad286b045 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:58.950606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bdbe7-a4d6-44ea-b8e5-c0fad286b045 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:58.982446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a37ae-6ad3-4322-969f-082d3f234f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:58.982989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a37ae-6ad3-4322-969f-082d3f234f69 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:58.983264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a37ae-6ad3-4322-969f-082d3f234f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:58.983533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a37ae-6ad3-4322-969f-082d3f234f69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:58.994287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a37ae-6ad3-4322-969f-082d3f234f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2d1a37ae-6ad3-4322-969f-082d3f234f69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:02:58.998853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:02:58.999933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:02:59.000825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 607/1195] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:02:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:02:59.005904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cf07f-11d7-4369-bbfb-f58fbd612359 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:59.103984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cf07f-11d7-4369-bbfb-f58fbd612359 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:59.136887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:59.137478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:59.137751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:59.138018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:59.138664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:59.138959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:59.140302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:59.148071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 589/1196] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:02:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:59.153745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a37ae-6ad3-4322-969f-082d3f234f69 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:59.171788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:59.254487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a37ae-6ad3-4322-969f-082d3f234f69 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:59.288109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023d45-08b4-40c6-aa58-7de2aa97c954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:59.289598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023d45-08b4-40c6-aa58-7de2aa97c954 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:59.289883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023d45-08b4-40c6-aa58-7de2aa97c954 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:59.290158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe023d45-08b4-40c6-aa58-7de2aa97c954 None admin] PATH_INFO: `/v3/users/d39b2fccd63c4e3bb53561fd9dceb5ae` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:59.291274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe023d45-08b4-40c6-aa58-7de2aa97c954 None admin] RBAC: Authorizing `identity:get_user(user_id=d39b2fccd63c4e3bb53561fd9dceb5ae)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:59.292733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe023d45-08b4-40c6-aa58-7de2aa97c954 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:59.294304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 608/1197] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Aug 9 19:02:59 2022] GET /identity/v3/users/d39b2fccd63c4e3bb53561fd9dceb5ae => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:59.300085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:59.356091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:59.401578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd93dc-37cb-4c64-9ce6-d75f84435ba3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:59.435010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6b73-42dd-4526-a99e-3ab1684880af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:59.435601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6b73-42dd-4526-a99e-3ab1684880af None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:59.435873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6b73-42dd-4526-a99e-3ab1684880af None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:59.436140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6b73-42dd-4526-a99e-3ab1684880af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:59.502033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b6b73-42dd-4526-a99e-3ab1684880af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:59.503503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b6b73-42dd-4526-a99e-3ab1684880af None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:59.557429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:59.599859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 590/1198] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:02:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:02:59.606355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023d45-08b4-40c6-aa58-7de2aa97c954 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:59.688395 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:02:59.703863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe023d45-08b4-40c6-aa58-7de2aa97c954 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:59.736481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bc593-7ed1-4618-a5e9-dba078b61e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:59.737100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bc593-7ed1-4618-a5e9-dba078b61e8d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:59.737375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bc593-7ed1-4618-a5e9-dba078b61e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:59.737645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bc593-7ed1-4618-a5e9-dba078b61e8d None admin] PATH_INFO: `/v3/projects/792a27137f804243866f534f2df2ed42` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:59.738736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bc593-7ed1-4618-a5e9-dba078b61e8d None admin] RBAC: Authorizing `identity:get_project(project_id=792a27137f804243866f534f2df2ed42)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:59.740166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bc593-7ed1-4618-a5e9-dba078b61e8d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:59.741954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 609/1199] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Aug 9 19:02:59 2022] GET /identity/v3/projects/792a27137f804243866f534f2df2ed42 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:02:59.748129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6b73-42dd-4526-a99e-3ab1684880af None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:02:59.848283 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6b73-42dd-4526-a99e-3ab1684880af None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:02:59.880985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1e10-07c9-491a-a1d9-9d1e77ba13e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:02:59.881576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1e10-07c9-491a-a1d9-9d1e77ba13e6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:02:59.881848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1e10-07c9-491a-a1d9-9d1e77ba13e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:02:59.882179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1e10-07c9-491a-a1d9-9d1e77ba13e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:02:59.882873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1e10-07c9-491a-a1d9-9d1e77ba13e6 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=792a27137f804243866f534f2df2ed42, user.id=d39b2fccd63c4e3bb53561fd9dceb5ae) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:02:59.883174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1e10-07c9-491a-a1d9-9d1e77ba13e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:02:59.884590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1e10-07c9-491a-a1d9-9d1e77ba13e6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:02:59.890047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 591/1200] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:02:59 2022] GET /identity/v3/role_assignments?user.id=d39b2fccd63c4e3bb53561fd9dceb5ae&scope.project.id=792a27137f804243866f534f2df2ed42&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:03:09.732366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=24 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:03:09.889191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:03:55.684394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfabd1e-dc62-4617-b8ef-6c962519e51a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:55.684394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfabd1e-dc62-4617-b8ef-6c962519e51a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:55.684394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfabd1e-dc62-4617-b8ef-6c962519e51a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:55.686709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 610/1201] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:03:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:03:55.692023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142100a2-0e72-4358-8a15-633dd6393559 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:55.692523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142100a2-0e72-4358-8a15-633dd6393559 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:55.692612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142100a2-0e72-4358-8a15-633dd6393559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:55.715337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-142100a2-0e72-4358-8a15-633dd6393559 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:03:55.716595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-142100a2-0e72-4358-8a15-633dd6393559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:03:55.796320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 592/1202] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:03:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:03:55.827813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfabd1e-dc62-4617-b8ef-6c962519e51a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:03:55.947622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfabd1e-dc62-4617-b8ef-6c962519e51a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:03:55.993895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:03:55.994683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:55.994907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:55.995182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:56.011993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:03:56.013499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:03:56.029439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:03:56.030572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:03:56.033694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:03:56.033694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:03:56.033694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:03:56.033694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 611/1203] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:03:56.040673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142100a2-0e72-4358-8a15-633dd6393559 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:03:56.184057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142100a2-0e72-4358-8a15-633dd6393559 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:03:56.231995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:03:56.240200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac8bae-0fe2-4873-944a-7d59a7ad6b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:03:56.241031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac8bae-0fe2-4873-944a-7d59a7ad6b9f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:56.241295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac8bae-0fe2-4873-944a-7d59a7ad6b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:56.241600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac8bae-0fe2-4873-944a-7d59a7ad6b9f None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:56.242341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac8bae-0fe2-4873-944a-7d59a7ad6b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:03:56.242716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac8bae-0fe2-4873-944a-7d59a7ad6b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:03:56.244267 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac8bae-0fe2-4873-944a-7d59a7ad6b9f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:03:56.260543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 593/1204] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:03:56.265616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:03:56.336616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:03:56.401219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57f993-88b7-4ec9-9352-f818c6b34d53 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:03:56.438883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04364de-e977-48a6-8cb9-7d9b625336c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:03:56.439646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04364de-e977-48a6-8cb9-7d9b625336c8 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:56.440026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04364de-e977-48a6-8cb9-7d9b625336c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:56.440418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04364de-e977-48a6-8cb9-7d9b625336c8 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:56.441169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04364de-e977-48a6-8cb9-7d9b625336c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:03:56.442693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04364de-e977-48a6-8cb9-7d9b625336c8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:03:56.466133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 612/1205] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Aug 9 19:03:56 2022] POST /identity/v3/users => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:03:58.913349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19818380-e137-43bd-bf26-3f66767cdef1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:58.913873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19818380-e137-43bd-bf26-3f66767cdef1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:58.913873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19818380-e137-43bd-bf26-3f66767cdef1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:58.915201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 594/1206] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:03:58.922683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def59f-ce4a-4911-8f6c-f84532ec327e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:58.923223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def59f-ce4a-4911-8f6c-f84532ec327e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:58.923223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def59f-ce4a-4911-8f6c-f84532ec327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:58.943599 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e2def59f-ce4a-4911-8f6c-f84532ec327e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:03:58.945263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e2def59f-ce4a-4911-8f6c-f84532ec327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:03:59.023164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 613/1207] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:03:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:03:59.047582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19818380-e137-43bd-bf26-3f66767cdef1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:03:59.192907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19818380-e137-43bd-bf26-3f66767cdef1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:03:59.242107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f215e7-c511-4582-821f-b3833353bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:03:59.242712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f215e7-c511-4582-821f-b3833353bb94 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:59.243007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f215e7-c511-4582-821f-b3833353bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:59.243286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f215e7-c511-4582-821f-b3833353bb94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:59.257880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f215e7-c511-4582-821f-b3833353bb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-14f215e7-c511-4582-821f-b3833353bb94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:03:59.262657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:03:59.264221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:03:59.266111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:03:59.268648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 595/1208] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:03:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:03:59.269967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2def59f-ce4a-4911-8f6c-f84532ec327e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:03:59.319957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:03:59.405902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2def59f-ce4a-4911-8f6c-f84532ec327e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:03:59.448245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:03:59.448812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:59.449264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:59.449543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:59.450210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:03:59.450516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:03:59.451899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:03:59.467351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 614/1209] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:03:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:03:59.468657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f215e7-c511-4582-821f-b3833353bb94 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:03:59.530276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:03:59.592642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f215e7-c511-4582-821f-b3833353bb94 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:03:59.640966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:03:59.641680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:59.641958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:59.642232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:59.655463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:03:59.656975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:03:59.676349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:03:59.677696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:03:59.678773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:03:59.678773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:03:59.678773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:03:59.678773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 596/1210] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:03:59.683987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:03:59.803040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:03:59.806485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52413d37-d9fc-44ab-b582-1dc59e3bbb78 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:03:59.854417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:03:59.855113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:03:59.855465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:03:59.855805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:03:59.856556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:03:59.856918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:03:59.858500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:03:59.872822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 615/1211] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:03:59.878809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:03:59.997444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a9612-046c-4f47-8b0b-5fc40e94cc08 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:03:59.999948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:00.036230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:00.036974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:00.037301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:00.037531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:00.045757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:00.047314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:04:00.060151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:00.061183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:00.062122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 597/1212] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:03:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:04:00.066449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:00.179499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403f537-8452-4580-ab08-c4876c3274b3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:00.219068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:00.219711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:00.219969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:00.220287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:00.221113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:00.221487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:00.223000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:00.250778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 616/1213] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Aug 9 19:04:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:00.257376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:00.385329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab633270-df0c-4c3f-b392-b1c54fdf4202 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:00.430148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816708a-8c84-4e8e-aa45-ce984329a9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:00.430802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816708a-8c84-4e8e-aa45-ce984329a9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:00.431083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816708a-8c84-4e8e-aa45-ce984329a9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:00.431357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816708a-8c84-4e8e-aa45-ce984329a9e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:00.473163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:00.518565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816708a-8c84-4e8e-aa45-ce984329a9e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:00.520123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816708a-8c84-4e8e-aa45-ce984329a9e1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:00.635734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 598/1214] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:04:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:04:00.641721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:00.662311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:00.765257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a75a-0e59-4cf5-867b-7e34eabf5c25 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:00.806403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:00.807128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:00.807405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:00.807674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:00.820540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:00.822035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:00.838348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:00.839734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:00.841160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:00.841160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:00.841160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:00.841160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 617/1215] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:00.846446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816708a-8c84-4e8e-aa45-ce984329a9e1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:00.918798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:00.963860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816708a-8c84-4e8e-aa45-ce984329a9e1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:01.001286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:01.002075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:01.003001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:01.003001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:01.003194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:01.003411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:01.004777 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:01.025359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 599/1216] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:01.026876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:01.107006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:01.149262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54245f59-67f0-4bd8-adee-0ac434b5945d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:01.195214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:01.195740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:01.196017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:01.196309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:01.203498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:01.205024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:04:01.218294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:01.219695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:01.221280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:04:01.221280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:04:01.221280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:04:01.221280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:04:01.221280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:04:01.221280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:04:01.221280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:01.221280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:01.229220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 618/1217] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:04:01.232045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:01.359315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85ae2e-faf4-454a-ba3b-383b317d448c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:01.393516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:01.407200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1e567-24cf-4836-80f9-2cc3de44a4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:01.408667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1e567-24cf-4836-80f9-2cc3de44a4ee None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:01.408667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1e567-24cf-4836-80f9-2cc3de44a4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:01.408667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1e567-24cf-4836-80f9-2cc3de44a4ee None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:01.409479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1e567-24cf-4836-80f9-2cc3de44a4ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:01.409838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1e567-24cf-4836-80f9-2cc3de44a4ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:01.411418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1e567-24cf-4836-80f9-2cc3de44a4ee None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:01.426643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 600/1218] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:01.433245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:01.542877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b4ab8-9f96-4195-9a88-7bd18bf00732 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:01.578082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:01.586561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b982d6-5550-46f1-8492-6e067bf66aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:01.587290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b982d6-5550-46f1-8492-6e067bf66aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:01.587762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b982d6-5550-46f1-8492-6e067bf66aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:01.588083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b982d6-5550-46f1-8492-6e067bf66aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:01.588947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b982d6-5550-46f1-8492-6e067bf66aa1 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=79078cd57816487e8b2b85856c13d5a8) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:01.589384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b982d6-5550-46f1-8492-6e067bf66aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:01.590997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b982d6-5550-46f1-8492-6e067bf66aa1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:01.597923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 619/1219] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:04:01 2022] GET /identity/v3/role_assignments?user.id=79078cd57816487e8b2b85856c13d5a8&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:03.700379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646a8eb-6f81-4cc7-9fe6-94e6dd55e5bd None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:03.700983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646a8eb-6f81-4cc7-9fe6-94e6dd55e5bd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:03.700983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646a8eb-6f81-4cc7-9fe6-94e6dd55e5bd None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:03.702465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 601/1220] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:03.710691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c74b20-6955-4a79-982a-c770e9bda72b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:03.711197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c74b20-6955-4a79-982a-c770e9bda72b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:03.711435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c74b20-6955-4a79-982a-c770e9bda72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:03.731401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-13c74b20-6955-4a79-982a-c770e9bda72b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:03.732806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-13c74b20-6955-4a79-982a-c770e9bda72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:03.809336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 620/1221] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 19:04:03.835249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646a8eb-6f81-4cc7-9fe6-94e6dd55e5bd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:03.945053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646a8eb-6f81-4cc7-9fe6-94e6dd55e5bd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:03.984097 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f540b2-d9a7-4526-a96c-ead43e12d78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:03.984678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f540b2-d9a7-4526-a96c-ead43e12d78a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:03.984952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f540b2-d9a7-4526-a96c-ead43e12d78a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:03.985219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f540b2-d9a7-4526-a96c-ead43e12d78a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:03.995942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f540b2-d9a7-4526-a96c-ead43e12d78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e2f540b2-d9a7-4526-a96c-ead43e12d78a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:04:04.000538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:04.001627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:04.002475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 602/1222] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:04:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:04:04.008322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c74b20-6955-4a79-982a-c770e9bda72b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:04.113121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c74b20-6955-4a79-982a-c770e9bda72b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:04.139571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:04.152426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:04.153061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:04.153329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:04.153601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:04.154270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:04.154570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:04.155955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:04.164902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 621/1223] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:04:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:04.170858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f540b2-d9a7-4526-a96c-ead43e12d78a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:04.215493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:04.280284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f540b2-d9a7-4526-a96c-ead43e12d78a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:04.315185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:04.315818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:04.316103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:04.316370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:04.329606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:04.331321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:04.347414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:04.353832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:04.354965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:04.354965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:04.354965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:04.354965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 603/1224] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:04:04.355256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:04.468781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7122d-f805-4d6f-909f-dacfea912189 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:04.506691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:04.507310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:04.507590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:04.507856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:04.508547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:04.508843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:04.510209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:04.523245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 622/1225] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:04.529280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:04.633174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:04.641375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f816-d572-483b-8c4b-71663e7e0117 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:04.685769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:04.686618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:04.687006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:04.687379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:04.698216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:04.699841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:04:04.713610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:04.715128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:04.716063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 604/1226] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:04.720935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:04.770314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:04.829985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401cd30-aff8-4c11-8e8c-a965595c90f5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:04.871637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:04.872238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:04.872537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:04.872806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:04.873523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:04.873822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:04.875239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:04.893283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 623/1227] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Aug 9 19:04:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:04.899788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:05.016255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040312-bd37-47d6-bd4b-5044b2ecf955 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:05.056664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:05.057424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:05.058036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:05.058372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:05.070594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:05.072158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:05.089079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:05.090292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:05.091218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:05.091218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:05.091218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:05.091218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 605/1228] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:05.096100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:05.213238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f168b8-f775-42d5-9dae-ecf9d3b66ea0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:05.215822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:05.253717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:05.254325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:05.254610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:05.254877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:05.256381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:05.256770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:05.258143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:05.270087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 624/1229] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:05.276821 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:05.288166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:05.393357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0089cf-1c04-4995-b531-f3e6b94973d3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:05.431776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:05.432577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:05.433129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:05.433462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:05.440964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:05.442968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:04:05.454753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:05.455877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:05.462310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 606/1230] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:04:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:05.462812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:05.578378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6776f-0d33-4911-a5d8-47469524a605 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:05.615915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c913db3-04e7-49af-bd13-d9a6a88d040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:05.616558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913db3-04e7-49af-bd13-d9a6a88d040d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:05.616843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913db3-04e7-49af-bd13-d9a6a88d040d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:05.617109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913db3-04e7-49af-bd13-d9a6a88d040d None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:05.617859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c913db3-04e7-49af-bd13-d9a6a88d040d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:05.618181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c913db3-04e7-49af-bd13-d9a6a88d040d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:05.619665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c913db3-04e7-49af-bd13-d9a6a88d040d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:05.632848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 625/1231] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:04:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:04:05.639012 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:05.707732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:05.747490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6b730-6984-4d91-8dfa-8b44d9444435 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:05.783955 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9ebb3-d0e0-4a9d-a1b6-e28e3cc91302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:05.784649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9ebb3-d0e0-4a9d-a1b6-e28e3cc91302 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:05.784931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9ebb3-d0e0-4a9d-a1b6-e28e3cc91302 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:05.785243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9ebb3-d0e0-4a9d-a1b6-e28e3cc91302 None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/79078cd57816487e8b2b85856c13d5a8/roles/72882169372f4db5bd4bf8c9c399b812` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:05.800144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9ebb3-d0e0-4a9d-a1b6-e28e3cc91302 None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=79078cd57816487e8b2b85856c13d5a8, role_id=72882169372f4db5bd4bf8c9c399b812)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:05.801632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9ebb3-d0e0-4a9d-a1b6-e28e3cc91302 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:05.816306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 607/1232] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:04:05 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/79078cd57816487e8b2b85856c13d5a8/roles/72882169372f4db5bd4bf8c9c399b812 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:04:07.970596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685de87-57e3-4c83-9687-72f9273ea0bb None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:07.971165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685de87-57e3-4c83-9687-72f9273ea0bb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:07.971165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685de87-57e3-4c83-9687-72f9273ea0bb None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:07.972495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 626/1233] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:07.978223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd17044-122d-4e50-a3d4-8e30b1821eb8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:07.978616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd17044-122d-4e50-a3d4-8e30b1821eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:07.978770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd17044-122d-4e50-a3d4-8e30b1821eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:07.996866 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-abd17044-122d-4e50-a3d4-8e30b1821eb8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:07.998227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-abd17044-122d-4e50-a3d4-8e30b1821eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:08.072566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 608/1234] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:04:08.098109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685de87-57e3-4c83-9687-72f9273ea0bb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:08.134116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:08.214306 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685de87-57e3-4c83-9687-72f9273ea0bb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:08.250070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ff01-f681-4aa0-8f51-ce4b27968ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:08.250624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ff01-f681-4aa0-8f51-ce4b27968ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:08.250898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ff01-f681-4aa0-8f51-ce4b27968ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:08.251165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ff01-f681-4aa0-8f51-ce4b27968ff9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:08.263397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277ff01-f681-4aa0-8f51-ce4b27968ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d277ff01-f681-4aa0-8f51-ce4b27968ff9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:04:08.268555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:08.269525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:08.270548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 627/1235] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Aug 9 19:04:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:08.276170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd17044-122d-4e50-a3d4-8e30b1821eb8 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:08.327924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:08.388816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd17044-122d-4e50-a3d4-8e30b1821eb8 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:08.426406 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:08.427665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:08.428032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:08.428348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:08.429040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:08.429406 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:08.430845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:08.440008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 609/1236] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Aug 9 19:04:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:08.446277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ff01-f681-4aa0-8f51-ce4b27968ff9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:08.541698 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:08.559763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ff01-f681-4aa0-8f51-ce4b27968ff9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:08.597506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:08.598230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:08.598543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:08.599667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:08.611221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:08.612757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:08.629511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:08.630682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:08.632251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:08.632251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:08.632251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:08.632251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 628/1237] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:04:08.637357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:08.752383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a0d7d-4782-486f-9798-6dd8d90d4021 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:08.763022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:08.791810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:08.792521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:08.792835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:08.793106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:08.793766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:08.794052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:08.795417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:08.807623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 610/1238] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:08.813530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:08.926127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3e438-9fcb-48cc-9de0-72910313995f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:08.970104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:08.970730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:08.971274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:08.971274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:08.974897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:08.980507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:08.982130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:04:08.994362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:08.995515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:08.996661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 629/1239] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:09.001452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:09.118196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eab6d6-f3b8-43d6-b7f5-e1b64a81ade2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:09.157592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:09.158197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:09.158470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:09.158739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:09.159460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:09.159761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:09.161852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:09.181287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 611/1240] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Aug 9 19:04:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:04:09.188362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:09.305777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891a7bc-966a-425d-baee-2dcd2f6a0eb8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:09.351030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95c3e3-7b3b-4627-a08f-b81ea6a972e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:09.351645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c3e3-7b3b-4627-a08f-b81ea6a972e0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:09.352003 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c3e3-7b3b-4627-a08f-b81ea6a972e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:09.352172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95c3e3-7b3b-4627-a08f-b81ea6a972e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:09.438665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:09.446647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95c3e3-7b3b-4627-a08f-b81ea6a972e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:09.448189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95c3e3-7b3b-4627-a08f-b81ea6a972e0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:09.574275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 630/1241] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:04:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:04:09.581539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:09.610565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:09.712179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c484cdc-8af9-4fbd-b2b0-133c30bf0cab None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:09.749513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:09.750303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:09.750710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:09.751085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:09.763271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:09.764757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:09.780780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:09.781885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:09.782857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:09.782857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:09.782857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:09.782857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 612/1242] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:09.788581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95c3e3-7b3b-4627-a08f-b81ea6a972e0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:09.894232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:09.901766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95c3e3-7b3b-4627-a08f-b81ea6a972e0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:09.946204 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:09.946998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:09.947280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:09.947557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:09.948309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:09.948706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:09.950447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:09.963369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 631/1243] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:09.969617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:10.073645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:10.101618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c6d83-f39d-4ccf-adb4-0a5d30a73bf8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:10.150229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:10.150889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:10.151176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:10.151509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:10.158788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:10.160225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:04:10.172254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:10.173871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:10.174964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 613/1244] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:04:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:10.180000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:10.301188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73c764-5119-4648-b690-87ae13453bae None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:10.343676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e3487-d102-452f-be38-c8d60f986dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:10.344248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e3487-d102-452f-be38-c8d60f986dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:10.344566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e3487-d102-452f-be38-c8d60f986dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:10.344823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e3487-d102-452f-be38-c8d60f986dc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:10.345529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e3487-d102-452f-be38-c8d60f986dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:10.345854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e3487-d102-452f-be38-c8d60f986dc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:10.347324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e3487-d102-452f-be38-c8d60f986dc9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:10.360144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 632/1245] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:04:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:10.369724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:10.482238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436e4aa-29e3-40d3-9d03-97a0cc996f78 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:10.521694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32791cad-e9e2-4056-b75f-e494e3684718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:10.522309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791cad-e9e2-4056-b75f-e494e3684718 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:10.522591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791cad-e9e2-4056-b75f-e494e3684718 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:10.522865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32791cad-e9e2-4056-b75f-e494e3684718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:10.523628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32791cad-e9e2-4056-b75f-e494e3684718 None admin] RBAC: Adding query filter params (role.id=72882169372f4db5bd4bf8c9c399b812, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=79078cd57816487e8b2b85856c13d5a8) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:10.524017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32791cad-e9e2-4056-b75f-e494e3684718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:10.525570 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32791cad-e9e2-4056-b75f-e494e3684718 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:10.531586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 614/1246] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:04:10 2022] GET /identity/v3/role_assignments?user.id=79078cd57816487e8b2b85856c13d5a8&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=72882169372f4db5bd4bf8c9c399b812 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:12.757353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc814a4a-b703-48b2-aad4-88ca9a619fc5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:12.757977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc814a4a-b703-48b2-aad4-88ca9a619fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:12.757977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc814a4a-b703-48b2-aad4-88ca9a619fc5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:12.759224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 633/1247] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:12.765282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a0d4b-f72d-4c32-94a0-fd5b81d5d31f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:12.765734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a0d4b-f72d-4c32-94a0-fd5b81d5d31f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:12.765842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a0d4b-f72d-4c32-94a0-fd5b81d5d31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:12.772694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:12.789432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-134a0d4b-f72d-4c32-94a0-fd5b81d5d31f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:12.790701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-134a0d4b-f72d-4c32-94a0-fd5b81d5d31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:12.869599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 615/1248] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:04:12.899714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc814a4a-b703-48b2-aad4-88ca9a619fc5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:12.921460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:13.022244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc814a4a-b703-48b2-aad4-88ca9a619fc5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:13.061586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9405-d6fc-4150-8947-d46e6cf32840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:13.062239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9405-d6fc-4150-8947-d46e6cf32840 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:13.062525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9405-d6fc-4150-8947-d46e6cf32840 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:13.062897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9405-d6fc-4150-8947-d46e6cf32840 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:13.075101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c9405-d6fc-4150-8947-d46e6cf32840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fc4c9405-d6fc-4150-8947-d46e6cf32840 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:04:13.080580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:13.081864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:13.083199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 634/1249] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:04:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:13.089531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a0d4b-f72d-4c32-94a0-fd5b81d5d31f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:13.211962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a0d4b-f72d-4c32-94a0-fd5b81d5d31f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:13.246509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:13.252925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:13.253540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:13.253814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:13.254081 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:13.254795 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:13.255555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:13.257226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:13.266772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 616/1250] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:13.274164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9405-d6fc-4150-8947-d46e6cf32840 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:13.361886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:13.388804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9405-d6fc-4150-8947-d46e6cf32840 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:13.426526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:13.427426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:13.427765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:13.428109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:13.443341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:13.444856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:13.461540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:13.462755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:13.463673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:13.463673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:13.463673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:13.463673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 635/1251] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:04:13.469333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:13.586252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab9fb6-ccf6-43d4-b39b-cff7068968a1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:13.627033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:13.627741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:13.628028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:13.628312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:13.629168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:13.629506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:13.631705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:13.645294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 617/1252] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:04:13.651838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:13.781940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b512b-5786-49ca-9ea4-31918b55a4f4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:13.811645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:13.821691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:13.822416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:13.822747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:13.823018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:13.831912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:13.834572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:04:13.847524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:13.848705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:13.850180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 636/1253] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:13.854992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:13.923935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:13.979221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986323c4-b8be-4970-97f0-a9a4f34e5341 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:14.022418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:14.023074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:14.023355 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:14.023629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:14.024365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:14.024713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:14.026310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:14.049228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 618/1254] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Aug 9 19:04:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:14.056135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:14.193764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e3082-b620-4b6f-a8cb-f9293eb31822 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:14.230604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a83fa-d606-491d-b661-ed48dfde6548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:14.231314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a83fa-d606-491d-b661-ed48dfde6548 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:14.231622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a83fa-d606-491d-b661-ed48dfde6548 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:14.231897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a83fa-d606-491d-b661-ed48dfde6548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:14.292908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:14.308278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a83fa-d606-491d-b661-ed48dfde6548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:14.309924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a83fa-d606-491d-b661-ed48dfde6548 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:14.429775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 637/1255] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:04:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:04:14.436287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:14.563064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3219d-c4a0-4486-b530-0e04e8348449 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:14.578071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:14.603107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:14.603778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:14.604088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:14.604404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:14.616787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:14.618196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:14.643673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:14.645008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:14.646666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:14.646666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:14.646666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:14.648376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 619/1256] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:14.654424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a83fa-d606-491d-b661-ed48dfde6548 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:14.752524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:14.769711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a83fa-d606-491d-b661-ed48dfde6548 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:14.815874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:14.816545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:14.816713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:14.816936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:14.817633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:14.817928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:14.819401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:14.831952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 638/1257] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:04:14.838565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:14.951655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c68845-b8fd-4847-85ab-c542595d9401 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:14.990474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:14.991276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:14.991553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:14.991836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:14.998913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:15.000346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:04:15.012495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:15.013638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:15.014666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 620/1258] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:15.020211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:15.152237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c1273-e500-4dc0-88bd-70c7e4600580 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:15.189357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b75566-a3cc-4fb0-90c2-27518619a541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:15.190363 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b75566-a3cc-4fb0-90c2-27518619a541 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:15.190661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b75566-a3cc-4fb0-90c2-27518619a541 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:15.190932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b75566-a3cc-4fb0-90c2-27518619a541 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:15.191624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b75566-a3cc-4fb0-90c2-27518619a541 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:15.191928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b75566-a3cc-4fb0-90c2-27518619a541 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:15.193446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b75566-a3cc-4fb0-90c2-27518619a541 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:15.206835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 639/1259] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:04:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:15.213236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:15.243664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:15.325330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ea49d-877f-413e-a5ed-e7274a97f47e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:15.361496 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e980a-72f7-473a-84d8-eaecc5cf848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:15.362475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e980a-72f7-473a-84d8-eaecc5cf848d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:15.362836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e980a-72f7-473a-84d8-eaecc5cf848d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:15.363169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e980a-72f7-473a-84d8-eaecc5cf848d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:15.363961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e980a-72f7-473a-84d8-eaecc5cf848d None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=79078cd57816487e8b2b85856c13d5a8) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:15.364897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e980a-72f7-473a-84d8-eaecc5cf848d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:15.366506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e980a-72f7-473a-84d8-eaecc5cf848d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:15.377727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 621/1260] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:04:15 2022] GET /identity/v3/role_assignments?user.id=79078cd57816487e8b2b85856c13d5a8&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:17.458865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4083-56f4-4974-bf7e-44ef910bf2dd None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:17.459550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4083-56f4-4974-bf7e-44ef910bf2dd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:17.459550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4083-56f4-4974-bf7e-44ef910bf2dd None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:17.460761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 640/1261] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:17.465794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c00ba-1622-4e51-9864-fe5aa2536488 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:17.466210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c00ba-1622-4e51-9864-fe5aa2536488 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:17.466326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c00ba-1622-4e51-9864-fe5aa2536488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:17.484070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d77c00ba-1622-4e51-9864-fe5aa2536488 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:17.485276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d77c00ba-1622-4e51-9864-fe5aa2536488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:17.549524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 622/1262] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:04:17.572465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed4083-56f4-4974-bf7e-44ef910bf2dd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:17.583822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:17.670828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed4083-56f4-4974-bf7e-44ef910bf2dd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:17.704583 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439d87-f1f9-4edc-948e-8a79cee67e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:17.705197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439d87-f1f9-4edc-948e-8a79cee67e36 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:17.705474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439d87-f1f9-4edc-948e-8a79cee67e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:17.705744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439d87-f1f9-4edc-948e-8a79cee67e36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:17.715608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93439d87-f1f9-4edc-948e-8a79cee67e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-93439d87-f1f9-4edc-948e-8a79cee67e36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:04:17.720251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:17.721355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:17.722235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 641/1263] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:04:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:17.727374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c00ba-1622-4e51-9864-fe5aa2536488 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:17.745810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:17.825815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c00ba-1622-4e51-9864-fe5aa2536488 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:17.858408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:17.859007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:17.859284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:17.859555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:17.860235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:17.860633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:17.861991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:17.870294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 623/1264] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:04:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:17.876264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439d87-f1f9-4edc-948e-8a79cee67e36 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:17.940484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:17.973241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439d87-f1f9-4edc-948e-8a79cee67e36 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:18.006608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:18.007224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:18.007501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:18.007764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:18.017635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:18.019058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:18.032939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:18.034034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:18.035127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:18.035127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:18.035127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:18.035127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 642/1265] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:18.040485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:18.130000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:18.152603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66818e44-aeb2-40d6-8e47-38c078dabf0e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:18.187483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:18.188079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:18.188350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:18.188643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:18.189270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:18.189567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:18.190963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:18.202153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 624/1266] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:18.207897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:18.305532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bab50-8779-4e44-a499-8068136531e1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:18.324760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:18.338691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:18.339303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:18.339581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:18.339856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:18.347007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:18.348987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:04:18.359965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:18.361065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:18.362197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 643/1267] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:18.367687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:18.468398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d703a7-9284-4db9-829b-b1e01543c1d4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:18.501911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:18.502509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:18.502787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:18.503057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:18.503774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:18.504080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:18.505842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:18.514190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:18.523198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 625/1268] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Aug 9 19:04:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:18.529052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:18.627201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e0f2a-7fe7-40c8-a91a-aa1d2d387f9d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:18.659687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:18.660294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:18.660608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:18.661005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:18.671624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:18.673103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:18.687220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:18.688297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:18.689237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:18.689237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:18.689237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:18.689237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 644/1269] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:18.694745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:18.790084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078de47f-cead-47f7-8fe6-24c6de566c37 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:18.823208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:18.823794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:18.824073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:18.824342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:18.825012 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:18.825310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:18.826650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:18.837340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 626/1270] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:18.843113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:18.874354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:18.940218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d44fe-3581-48be-84a0-ec742937a4db None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:18.971453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:18.972052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:18.972322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:18.972614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:18.979470 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:18.980945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:04:18.992799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:18.993863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:18.994988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 645/1271] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:04:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:18.999793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:19.057771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:19.103394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6a062-e09e-4d5e-a05d-ace048e8bc16 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:19.142399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10a798-78fb-4e5b-ba2e-559dbc380f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:19.142994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a798-78fb-4e5b-ba2e-559dbc380f2d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:19.143265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a798-78fb-4e5b-ba2e-559dbc380f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:19.143531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a798-78fb-4e5b-ba2e-559dbc380f2d None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:19.144204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10a798-78fb-4e5b-ba2e-559dbc380f2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:19.144529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10a798-78fb-4e5b-ba2e-559dbc380f2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:19.146742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10a798-78fb-4e5b-ba2e-559dbc380f2d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:19.158985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 627/1272] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:04:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:19.165759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:19.253277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:19.261681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e27a1-6e36-4f97-af71-7fa93fbf415c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:19.294778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c4187-1b62-43db-900d-e8255c723ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:19.295425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c4187-1b62-43db-900d-e8255c723ffc None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:19.295705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c4187-1b62-43db-900d-e8255c723ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:19.295977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c4187-1b62-43db-900d-e8255c723ffc None admin] PATH_INFO: `/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/79078cd57816487e8b2b85856c13d5a8/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:19.297880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c4187-1b62-43db-900d-e8255c723ffc None admin] RBAC: Authorizing `identity:create_grant(project_id=32d2105c511b40ed9df707fde095a0b6, user_id=79078cd57816487e8b2b85856c13d5a8, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:19.299308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c4187-1b62-43db-900d-e8255c723ffc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:19.312083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 646/1273] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Aug 9 19:04:19 2022] PUT /identity/v3/projects/32d2105c511b40ed9df707fde095a0b6/users/79078cd57816487e8b2b85856c13d5a8/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:04:21.336417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de0c6c-2dfd-4d9b-8e24-7191c29715ba None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:21.337010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de0c6c-2dfd-4d9b-8e24-7191c29715ba None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:21.337010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de0c6c-2dfd-4d9b-8e24-7191c29715ba None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:21.338302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 628/1274] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:21.343451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898872a1-d5e8-4b22-b243-a9f1e68c3bcb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:21.343859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898872a1-d5e8-4b22-b243-a9f1e68c3bcb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:21.344001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898872a1-d5e8-4b22-b243-a9f1e68c3bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:21.362231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-898872a1-d5e8-4b22-b243-a9f1e68c3bcb None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:21.363404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-898872a1-d5e8-4b22-b243-a9f1e68c3bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:21.431514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 647/1275] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:04:21.454915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de0c6c-2dfd-4d9b-8e24-7191c29715ba None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:21.555862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de0c6c-2dfd-4d9b-8e24-7191c29715ba None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:21.573301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:21.591608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d060a-e68e-44f7-9b11-e72ec3e6ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:21.592615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d060a-e68e-44f7-9b11-e72ec3e6ce61 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:21.593085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d060a-e68e-44f7-9b11-e72ec3e6ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:21.593572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d060a-e68e-44f7-9b11-e72ec3e6ce61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:21.605806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d060a-e68e-44f7-9b11-e72ec3e6ce61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-336d060a-e68e-44f7-9b11-e72ec3e6ce61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 19:04:21.610439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:21.611641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:21.612547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 629/1276] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Aug 9 19:04:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:21.618284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898872a1-d5e8-4b22-b243-a9f1e68c3bcb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:21.700224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:21.718991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898872a1-d5e8-4b22-b243-a9f1e68c3bcb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:21.752744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:21.753341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:21.753619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:21.753891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:21.755148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:21.755657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:21.757234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:21.765365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 648/1277] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Aug 9 19:04:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:21.771238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d060a-e68e-44f7-9b11-e72ec3e6ce61 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:21.901260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d060a-e68e-44f7-9b11-e72ec3e6ce61 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:21.946581 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:21.947423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:21.947805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:21.948177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:21.961775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:21.963849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:21.983584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:21.984770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:21.990743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:21.990743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:21.990743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:21.990743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 630/1278] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 19:04:21.991142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:22.098330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cfb34-b7cf-4aa0-a65d-b2fc4e20f7e9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:22.135337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:22.135934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:22.136212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:22.136510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:22.137148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:22.137450 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:22.138883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:22.151544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 649/1279] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:22.157206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:22.187978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:22.255539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa1010-be5f-4978-932a-942652b7d44b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:22.287891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:22.288525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:22.288799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:22.289063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:22.296096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:22.297540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 19:04:22.309163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:22.310318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:22.311501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 631/1280] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:22.316196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:22.320561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:22.420476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035094a-6125-462e-ac72-39fc4e90a01e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:22.455018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:22.455607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:22.455885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:22.456154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:22.456899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:22.457205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:22.459016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:22.475774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 650/1281] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Aug 9 19:04:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:22.481691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:22.573605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:22.583808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbafe4-1fed-418a-9fa3-7e4a0590915f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:22.619660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8585de-be47-47cd-b0c1-58c7d864b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:22.621425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8585de-be47-47cd-b0c1-58c7d864b99e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:22.621425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8585de-be47-47cd-b0c1-58c7d864b99e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:22.621425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8585de-be47-47cd-b0c1-58c7d864b99e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:22.690794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8585de-be47-47cd-b0c1-58c7d864b99e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:22.692383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8585de-be47-47cd-b0c1-58c7d864b99e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:22.779948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:22.791149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 632/1282] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Aug 9 19:04:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 19:04:22.798485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:22.867256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:22.906719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc095b51-9a20-440d-8389-5704cad04e9d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:22.940679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:22.941354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:22.941581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:22.941853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:22.952389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:22.953856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 19:04:22.968149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:22.969259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 19:04:22.970311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 19:04:22.970311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 19:04:22.970311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:22.970311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 651/1283] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:22.975299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8585de-be47-47cd-b0c1-58c7d864b99e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:23.074604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8585de-be47-47cd-b0c1-58c7d864b99e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:23.114576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:23.115285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:23.115557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:23.115823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:23.116485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:23.116806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:23.118140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:23.129100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 633/1284] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Aug 9 19:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:23.136984 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:23.237280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e128-33ba-4ce9-b29f-1bdee7b8b408 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:23.250911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:23.271327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:23.271942 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:23.272222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:23.272525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:23.278970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:23.280815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 19:04:23.292270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:23.297992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Aug 09 19:04:23.299196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 652/1285] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Aug 9 19:04:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:23.299853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:23.351128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:23.401645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e8b1d-9d22-46a8-9de7-2b9ebbdaadf0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:23.436411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6657ce-253b-4496-9ab6-569d91fec65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:23.437040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6657ce-253b-4496-9ab6-569d91fec65f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:23.437320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6657ce-253b-4496-9ab6-569d91fec65f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:23.437594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6657ce-253b-4496-9ab6-569d91fec65f None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:23.438274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6657ce-253b-4496-9ab6-569d91fec65f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:23.438579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6657ce-253b-4496-9ab6-569d91fec65f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:23.439985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6657ce-253b-4496-9ab6-569d91fec65f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:23.451735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 634/1286] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Aug 9 19:04:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:04:23.458080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:23.556000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23b37e-42e7-4b7c-a04c-c34d82f9ddea None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:23.588732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1296b27-7897-4db1-a819-86639ddbc5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:23.589331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1296b27-7897-4db1-a819-86639ddbc5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:23.589609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1296b27-7897-4db1-a819-86639ddbc5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:23.589881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1296b27-7897-4db1-a819-86639ddbc5d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:23.590596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1296b27-7897-4db1-a819-86639ddbc5d2 None admin] RBAC: Adding query filter params (role.id=38ea9e633951415f8b0c0c663e24e8b7, scope.project.id=32d2105c511b40ed9df707fde095a0b6, user.id=79078cd57816487e8b2b85856c13d5a8) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:23.590902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1296b27-7897-4db1-a819-86639ddbc5d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:23.592338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1296b27-7897-4db1-a819-86639ddbc5d2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:23.597954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 653/1287] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Aug 9 19:04:23 2022] GET /identity/v3/role_assignments?user.id=79078cd57816487e8b2b85856c13d5a8&scope.project.id=32d2105c511b40ed9df707fde095a0b6&role.id=38ea9e633951415f8b0c0c663e24e8b7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:25.684869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5f19-98ed-47df-8358-e0a1fab244b0 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:25.685417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5f19-98ed-47df-8358-e0a1fab244b0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:25.685417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b5f19-98ed-47df-8358-e0a1fab244b0 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:25.686742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 635/1288] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:25.691850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16672da-edba-429b-8a52-4b2de2e15cfd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:25.692831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16672da-edba-429b-8a52-4b2de2e15cfd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:25.693031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16672da-edba-429b-8a52-4b2de2e15cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:25.712358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f16672da-edba-429b-8a52-4b2de2e15cfd None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:25.713586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f16672da-edba-429b-8a52-4b2de2e15cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:25.741678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:25.778444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 654/1289] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:04:25.800363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b5f19-98ed-47df-8358-e0a1fab244b0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:25.917498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b5f19-98ed-47df-8358-e0a1fab244b0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:25.928751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:25.953055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:25.953612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:25.953886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:25.954156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:25.954729 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:25.956088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:25.964711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:25.965988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:25.967058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 636/1290] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:04:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:25.971938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16672da-edba-429b-8a52-4b2de2e15cfd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:26.072170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16672da-edba-429b-8a52-4b2de2e15cfd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:26.112389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c413f2-dbe6-4bd4-8c6a-bcc05f4c946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:26.113346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c413f2-dbe6-4bd4-8c6a-bcc05f4c946b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:26.113655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c413f2-dbe6-4bd4-8c6a-bcc05f4c946b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:26.114697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c413f2-dbe6-4bd4-8c6a-bcc05f4c946b None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:26.115739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c413f2-dbe6-4bd4-8c6a-bcc05f4c946b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:26.116268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c413f2-dbe6-4bd4-8c6a-bcc05f4c946b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:26.118435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c413f2-dbe6-4bd4-8c6a-bcc05f4c946b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:26.125541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 655/1291] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Aug 9 19:04:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:26.132737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:26.240071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb7433-5dac-49be-976d-90594bd31c04 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:26.272719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64d6c2-72ab-4ffe-98b5-a886b815ccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:26.273312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64d6c2-72ab-4ffe-98b5-a886b815ccb6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:26.273585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64d6c2-72ab-4ffe-98b5-a886b815ccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:26.273852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64d6c2-72ab-4ffe-98b5-a886b815ccb6 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:26.274467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64d6c2-72ab-4ffe-98b5-a886b815ccb6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:26.274765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64d6c2-72ab-4ffe-98b5-a886b815ccb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:26.276117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64d6c2-72ab-4ffe-98b5-a886b815ccb6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:26.280393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 637/1292] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Aug 9 19:04:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:28.336338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678558-adda-4b9c-bf45-396a027247b1 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:28.336895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678558-adda-4b9c-bf45-396a027247b1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:28.336895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678558-adda-4b9c-bf45-396a027247b1 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:28.338243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 656/1293] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:28.343547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dce526-b679-436c-88c7-9dc1104f3c47 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:28.343984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dce526-b679-436c-88c7-9dc1104f3c47 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:28.344089 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dce526-b679-436c-88c7-9dc1104f3c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:28.362543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-77dce526-b679-436c-88c7-9dc1104f3c47 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:28.364278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-77dce526-b679-436c-88c7-9dc1104f3c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:28.393332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:28.431744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 638/1294] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:04:28.455631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678558-adda-4b9c-bf45-396a027247b1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:28.486716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:28.556573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678558-adda-4b9c-bf45-396a027247b1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:28.588955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cb9bd-bb22-4764-869e-3cedfa6f3376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:28.589482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb9bd-bb22-4764-869e-3cedfa6f3376 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:28.589759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb9bd-bb22-4764-869e-3cedfa6f3376 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:28.590028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb9bd-bb22-4764-869e-3cedfa6f3376 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:28.590666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cb9bd-bb22-4764-869e-3cedfa6f3376 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:28.592015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cb9bd-bb22-4764-869e-3cedfa6f3376 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:28.608241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 657/1295] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Aug 9 19:04:28 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:04:30.685957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b4137-4d9a-4515-bb51-428e9262b3a9 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:30.686525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b4137-4d9a-4515-bb51-428e9262b3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:30.686525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b4137-4d9a-4515-bb51-428e9262b3a9 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:30.687798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 639/1296] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:30.692911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a446e0b-8f1e-40e2-b5cd-6ca247bbad68 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:30.693331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a446e0b-8f1e-40e2-b5cd-6ca247bbad68 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:30.693450 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a446e0b-8f1e-40e2-b5cd-6ca247bbad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:30.711839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1a446e0b-8f1e-40e2-b5cd-6ca247bbad68 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:30.713055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1a446e0b-8f1e-40e2-b5cd-6ca247bbad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:30.783057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 658/1297] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:04:30.807236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b4137-4d9a-4515-bb51-428e9262b3a9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:30.905580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b4137-4d9a-4515-bb51-428e9262b3a9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:30.939467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:30.940020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:30.940298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:30.940600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:30.941174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:30.942734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:30.951131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:30.952388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:30.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 640/1298] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:04:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:30.958449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a446e0b-8f1e-40e2-b5cd-6ca247bbad68 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:30.977288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:31.057896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a446e0b-8f1e-40e2-b5cd-6ca247bbad68 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:31.092964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:31.093600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:31.094008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:31.094316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:31.095317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:31.095685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:31.097465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:31.105054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 659/1299] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Aug 9 19:04:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:31.111248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:31.147059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:31.218378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78bc3-1c36-454f-aea9-55b4c928e64c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:31.251351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569beaa-3052-41d3-ac3b-47a9931bc864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:31.251948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569beaa-3052-41d3-ac3b-47a9931bc864 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:31.252232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569beaa-3052-41d3-ac3b-47a9931bc864 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:31.252529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569beaa-3052-41d3-ac3b-47a9931bc864 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:31.253222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569beaa-3052-41d3-ac3b-47a9931bc864 None admin] RBAC: Adding query filter params (interface=public, service_id=447f588e83e747139a00f8096dbdd8a7, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:31.253530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569beaa-3052-41d3-ac3b-47a9931bc864 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:31.255100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569beaa-3052-41d3-ac3b-47a9931bc864 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:31.259931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 641/1300] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Aug 9 19:04:31 2022] GET /identity/v3/endpoints?service_id=447f588e83e747139a00f8096dbdd8a7&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:31.265600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:31.344320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:31.362349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a1e6e-baf8-4451-914e-f219dc2f3a5a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:31.395618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d45f-eb3a-4834-9496-9d40d800a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:31.396215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d45f-eb3a-4834-9496-9d40d800a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:31.396521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d45f-eb3a-4834-9496-9d40d800a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:31.396792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d45f-eb3a-4834-9496-9d40d800a7f6 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:31.397392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56d45f-eb3a-4834-9496-9d40d800a7f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:31.398748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56d45f-eb3a-4834-9496-9d40d800a7f6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:31.405386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 660/1301] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Aug 9 19:04:31 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:04:33.412867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d657e-4cb6-456a-a776-e53ac40aceb4 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:33.413325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d657e-4cb6-456a-a776-e53ac40aceb4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:33.413508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d657e-4cb6-456a-a776-e53ac40aceb4 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:33.414766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 642/1302] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:04:33.419912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bdaff-c74f-4489-af88-ffae47e1583f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:33.420325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bdaff-c74f-4489-af88-ffae47e1583f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:33.420493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bdaff-c74f-4489-af88-ffae47e1583f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:33.439562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cb3bdaff-c74f-4489-af88-ffae47e1583f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:04:33.440777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cb3bdaff-c74f-4489-af88-ffae47e1583f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:04:33.503894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 661/1303] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:04:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 19:04:33.526713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d657e-4cb6-456a-a776-e53ac40aceb4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:33.619778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:33.626587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d657e-4cb6-456a-a776-e53ac40aceb4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:33.659439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:33.660047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:33.660319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:33.660760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:33.661332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:33.662659 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 19:04:33.670982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 19:04:33.672167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Aug 09 19:04:33.673061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 643/1304] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Aug 9 19:04:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 19:04:33.679018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bdaff-c74f-4489-af88-ffae47e1583f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:33.756614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:33.778693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bdaff-c74f-4489-af88-ffae47e1583f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:33.811675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1140bc-3bc7-48e8-802f-29a497c2599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:33.812262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1140bc-3bc7-48e8-802f-29a497c2599d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:33.812563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1140bc-3bc7-48e8-802f-29a497c2599d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:33.812831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1140bc-3bc7-48e8-802f-29a497c2599d None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:33.813446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1140bc-3bc7-48e8-802f-29a497c2599d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:04:33.813742 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1140bc-3bc7-48e8-802f-29a497c2599d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:33.815091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1140bc-3bc7-48e8-802f-29a497c2599d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:33.820523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 662/1305] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Aug 9 19:04:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:04:33.826277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:04:33.923211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7770b-3e4c-4143-82ac-351587a17bdd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:04:33.955918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061637c-9ab5-42cc-bd23-c8857441cf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:04:33.956548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061637c-9ab5-42cc-bd23-c8857441cf9f None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:04:33.956829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061637c-9ab5-42cc-bd23-c8857441cf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:04:33.957103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061637c-9ab5-42cc-bd23-c8857441cf9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:04:33.957803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061637c-9ab5-42cc-bd23-c8857441cf9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:04:33.959134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061637c-9ab5-42cc-bd23-c8857441cf9f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:04:33.978429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 644/1306] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:04:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:04:43.817589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=30 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:04:43.974127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:38.015633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bdd0d-e67a-47d2-89b6-cd23a21a12b4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:38.016663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bdd0d-e67a-47d2-89b6-cd23a21a12b4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:38.017134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bdd0d-e67a-47d2-89b6-cd23a21a12b4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:38.020527 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 663/1307] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:05:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:38.025984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674d1d-0e9c-490f-bfd8-c2770add0e21 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:38.026488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674d1d-0e9c-490f-bfd8-c2770add0e21 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:38.026572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674d1d-0e9c-490f-bfd8-c2770add0e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:38.126979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-60674d1d-0e9c-490f-bfd8-c2770add0e21 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:38.130213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-60674d1d-0e9c-490f-bfd8-c2770add0e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:38.275656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 645/1308] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:38.329272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fccef-74d4-408b-97c6-f634a7f903be None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:38.329858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fccef-74d4-408b-97c6-f634a7f903be None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:38.330319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fccef-74d4-408b-97c6-f634a7f903be None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:38.332051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 664/1309] 10.222.0.70 () {54 vars in 965 bytes} [Tue Aug 9 19:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:38.338678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2717c6-8068-4f47-8c1f-d819af6aac7d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:38.338956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2717c6-8068-4f47-8c1f-d819af6aac7d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:38.339231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2717c6-8068-4f47-8c1f-d819af6aac7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:38.434882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1d2717c6-8068-4f47-8c1f-d819af6aac7d None None] MFA Rules not processed for user `31183ac0fdd642e6a1a7578899aa9fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:38.436387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1d2717c6-8068-4f47-8c1f-d819af6aac7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:38.459090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:38.522657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 646/1310] 10.222.0.70 () {58 vars in 1062 bytes} [Tue Aug 9 19:05:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:05:38.531427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fccef-74d4-408b-97c6-f634a7f903be None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:38.533194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-248fccef-74d4-408b-97c6-f634a7f903be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:38.650156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fccef-74d4-408b-97c6-f634a7f903be None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:38.735878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6bc5b-cc7e-46d8-b902-67d7cf49a00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:38.736788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6bc5b-cc7e-46d8-b902-67d7cf49a00e service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:38.737213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6bc5b-cc7e-46d8-b902-67d7cf49a00e service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:38.737741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6bc5b-cc7e-46d8-b902-67d7cf49a00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:38.740836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dda6bc5b-cc7e-46d8-b902-67d7cf49a00e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:38.811652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:38.885538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6bc5b-cc7e-46d8-b902-67d7cf49a00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:38.887303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6bc5b-cc7e-46d8-b902-67d7cf49a00e service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:38.922481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 665/1311] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:05:41.310030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983a6a-613a-480f-87c7-4bc283c8637b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:41.310549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983a6a-613a-480f-87c7-4bc283c8637b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:41.310549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983a6a-613a-480f-87c7-4bc283c8637b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:41.312039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 647/1312] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:41.318098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c43c1-ceeb-44bc-94ff-6501cf271bb9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:41.318606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c43c1-ceeb-44bc-94ff-6501cf271bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:41.318713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c43c1-ceeb-44bc-94ff-6501cf271bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:41.405301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7f0c43c1-ceeb-44bc-94ff-6501cf271bb9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:41.406543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7f0c43c1-ceeb-44bc-94ff-6501cf271bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:41.511212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 666/1313] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:41.565246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b9c61-fdf4-41f2-9458-5fe80fb1b6c0 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:41.565450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b9c61-fdf4-41f2-9458-5fe80fb1b6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:41.565726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b9c61-fdf4-41f2-9458-5fe80fb1b6c0 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:41.567728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 648/1314] 10.222.0.70 () {54 vars in 965 bytes} [Tue Aug 9 19:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:41.573876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5543bc-d2d2-420f-b0d8-e1f4f04b379e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:41.574169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5543bc-d2d2-420f-b0d8-e1f4f04b379e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:41.574448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5543bc-d2d2-420f-b0d8-e1f4f04b379e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:41.586120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:41.653130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4b5543bc-d2d2-420f-b0d8-e1f4f04b379e None None] MFA Rules not processed for user `31183ac0fdd642e6a1a7578899aa9fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:41.654328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4b5543bc-d2d2-420f-b0d8-e1f4f04b379e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:41.733046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 667/1315] 10.222.0.70 () {58 vars in 1062 bytes} [Tue Aug 9 19:05:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:41.742767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b9c61-fdf4-41f2-9458-5fe80fb1b6c0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:41.744318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-520b9c61-fdf4-41f2-9458-5fe80fb1b6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:41.848426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b9c61-fdf4-41f2-9458-5fe80fb1b6c0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:41.918661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:41.921581 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8e8c-bf51-409c-87a1-ecbdb3f8c994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:41.922194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8e8c-bf51-409c-87a1-ecbdb3f8c994 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:41.922626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8e8c-bf51-409c-87a1-ecbdb3f8c994 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:41.923070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8e8c-bf51-409c-87a1-ecbdb3f8c994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:41.924710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0f0f8e8c-bf51-409c-87a1-ecbdb3f8c994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:42.031693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8e8c-bf51-409c-87a1-ecbdb3f8c994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:42.033153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8e8c-bf51-409c-87a1-ecbdb3f8c994 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:42.050561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 649/1316] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:05:44.638353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47ed8c-fdb0-49ac-9cfe-7b6b09c32cd7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:44.638856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47ed8c-fdb0-49ac-9cfe-7b6b09c32cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:44.638856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47ed8c-fdb0-49ac-9cfe-7b6b09c32cd7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:44.640169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 668/1317] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:44.645514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda0866-0214-4607-9089-30199bac10e8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:44.645931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda0866-0214-4607-9089-30199bac10e8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:44.646052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda0866-0214-4607-9089-30199bac10e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:44.710771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6fda0866-0214-4607-9089-30199bac10e8 None None] MFA Rules not processed for user `832165b8104e4cafa72e1d439a9295d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:44.712009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6fda0866-0214-4607-9089-30199bac10e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:44.768701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:44.813630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 650/1318] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:05:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:44.854870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47ed8c-fdb0-49ac-9cfe-7b6b09c32cd7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:44.856405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6f47ed8c-fdb0-49ac-9cfe-7b6b09c32cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:44.965668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47ed8c-fdb0-49ac-9cfe-7b6b09c32cd7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:44.968465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:45.046777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb41b3-2ded-43bb-9530-e5e775f4aa83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:45.047381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb41b3-2ded-43bb-9530-e5e775f4aa83 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:45.047655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb41b3-2ded-43bb-9530-e5e775f4aa83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:45.047923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb41b3-2ded-43bb-9530-e5e775f4aa83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:45.049576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-68bb41b3-2ded-43bb-9530-e5e775f4aa83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:45.160090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb41b3-2ded-43bb-9530-e5e775f4aa83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:45.161872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb41b3-2ded-43bb-9530-e5e775f4aa83 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:45.179907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 669/1319] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:05:47.976130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4a48f-3147-42e9-9f57-b5143cc21b37 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:47.976688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4a48f-3147-42e9-9f57-b5143cc21b37 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:47.976688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4a48f-3147-42e9-9f57-b5143cc21b37 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:47.977946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 651/1320] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:47.983055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917beaab-31f0-40cf-b8ba-b1433b093de9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:47.983462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917beaab-31f0-40cf-b8ba-b1433b093de9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:47.983677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917beaab-31f0-40cf-b8ba-b1433b093de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:48.032458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:48.058854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-917beaab-31f0-40cf-b8ba-b1433b093de9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:48.060064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-917beaab-31f0-40cf-b8ba-b1433b093de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:48.157687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 670/1321] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:05:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:48.197377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4a48f-3147-42e9-9f57-b5143cc21b37 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:48.198924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b3a4a48f-3147-42e9-9f57-b5143cc21b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:48.299039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4a48f-3147-42e9-9f57-b5143cc21b37 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:48.370354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c92ca-b5ab-46a1-afb5-158da35937c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:48.370947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c92ca-b5ab-46a1-afb5-158da35937c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:48.371221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c92ca-b5ab-46a1-afb5-158da35937c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:48.371488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c92ca-b5ab-46a1-afb5-158da35937c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:48.373105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2d3c92ca-b5ab-46a1-afb5-158da35937c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:48.381858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:48.486261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c92ca-b5ab-46a1-afb5-158da35937c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:48.487680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c92ca-b5ab-46a1-afb5-158da35937c6 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:48.505734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 652/1322] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:05:51.235629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556382f-2155-4829-a29e-dacf000b5eba None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:51.236134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556382f-2155-4829-a29e-dacf000b5eba None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:51.236134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556382f-2155-4829-a29e-dacf000b5eba None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:51.237453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 671/1323] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:05:51.243324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26125309-73ff-457a-854d-67f32bd61848 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:51.243785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26125309-73ff-457a-854d-67f32bd61848 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:51.243903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26125309-73ff-457a-854d-67f32bd61848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:51.309869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-26125309-73ff-457a-854d-67f32bd61848 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:51.311064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-26125309-73ff-457a-854d-67f32bd61848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:51.371895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:51.419380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 653/1324] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:51.457417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556382f-2155-4829-a29e-dacf000b5eba None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:51.458956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d556382f-2155-4829-a29e-dacf000b5eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:51.571786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556382f-2155-4829-a29e-dacf000b5eba None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:51.584063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:51.646918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517d903-be3a-412f-98b3-7aa921f227e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:51.647821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d903-be3a-412f-98b3-7aa921f227e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:51.647821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d903-be3a-412f-98b3-7aa921f227e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:51.648036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d903-be3a-412f-98b3-7aa921f227e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:51.649682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6517d903-be3a-412f-98b3-7aa921f227e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:51.759962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517d903-be3a-412f-98b3-7aa921f227e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:51.761418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517d903-be3a-412f-98b3-7aa921f227e0 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:51.779665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 672/1325] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:05:53.856679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1b685-9dc7-484e-b732-8739ac7ca0ae None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:53.857214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1b685-9dc7-484e-b732-8739ac7ca0ae None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:53.857214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1b685-9dc7-484e-b732-8739ac7ca0ae None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:53.858548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 654/1326] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:53.864096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab028c-bb21-4344-87db-6fd4ec657122 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:53.864571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab028c-bb21-4344-87db-6fd4ec657122 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:53.864571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab028c-bb21-4344-87db-6fd4ec657122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:53.914909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:53.935049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e0ab028c-bb21-4344-87db-6fd4ec657122 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:53.936292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e0ab028c-bb21-4344-87db-6fd4ec657122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:54.031861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 673/1327] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:54.069982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1b685-9dc7-484e-b732-8739ac7ca0ae None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:54.071536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4fb1b685-9dc7-484e-b732-8739ac7ca0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:54.177259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1b685-9dc7-484e-b732-8739ac7ca0ae None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:54.252892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fe6f0-a5c8-48d2-aba9-470fb480be47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:54.253499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fe6f0-a5c8-48d2-aba9-470fb480be47 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:54.254368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fe6f0-a5c8-48d2-aba9-470fb480be47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:54.254788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fe6f0-a5c8-48d2-aba9-470fb480be47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:54.256597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1d5fe6f0-a5c8-48d2-aba9-470fb480be47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:54.273207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:54.372474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5fe6f0-a5c8-48d2-aba9-470fb480be47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:54.373819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5fe6f0-a5c8-48d2-aba9-470fb480be47 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:54.392754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 655/1328] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:05:56.521941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9261a1d-11b9-498b-9e76-c807d8a6c1cc None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:56.522541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9261a1d-11b9-498b-9e76-c807d8a6c1cc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:56.522541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9261a1d-11b9-498b-9e76-c807d8a6c1cc None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:56.524018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 674/1329] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:56.529290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bb526-080f-424a-bdaf-fb7a4dc2d3fb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:56.529698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bb526-080f-424a-bdaf-fb7a4dc2d3fb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:56.529845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bb526-080f-424a-bdaf-fb7a4dc2d3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:56.595665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-016bb526-080f-424a-bdaf-fb7a4dc2d3fb None None] MFA Rules not processed for user `832165b8104e4cafa72e1d439a9295d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:56.596908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-016bb526-080f-424a-bdaf-fb7a4dc2d3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:56.664315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:56.698944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 656/1330] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:05:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:56.738194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9261a1d-11b9-498b-9e76-c807d8a6c1cc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:56.740747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b9261a1d-11b9-498b-9e76-c807d8a6c1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:56.844793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9261a1d-11b9-498b-9e76-c807d8a6c1cc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:56.864084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:56.919231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d0350-82eb-46c3-9689-6b97cd32b840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:56.919840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d0350-82eb-46c3-9689-6b97cd32b840 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:56.920117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d0350-82eb-46c3-9689-6b97cd32b840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:56.920387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d0350-82eb-46c3-9689-6b97cd32b840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:56.922080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b83d0350-82eb-46c3-9689-6b97cd32b840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:57.038983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d0350-82eb-46c3-9689-6b97cd32b840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:57.040431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d0350-82eb-46c3-9689-6b97cd32b840 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:57.059027 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 675/1331] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:05:59.378828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b913b-28d2-421f-84de-d0636017bfbf None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:59.379400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b913b-28d2-421f-84de-d0636017bfbf None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:59.379400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b913b-28d2-421f-84de-d0636017bfbf None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:59.380645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 657/1332] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:05:59.385840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdbb0d-efc6-49ce-b600-4aee3367f889 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:59.386267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdbb0d-efc6-49ce-b600-4aee3367f889 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:59.386481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdbb0d-efc6-49ce-b600-4aee3367f889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:59.448914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:59.451713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-43cdbb0d-efc6-49ce-b600-4aee3367f889 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:05:59.453589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-43cdbb0d-efc6-49ce-b600-4aee3367f889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:59.551797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 676/1333] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:05:59.588577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b913b-28d2-421f-84de-d0636017bfbf None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:05:59.590142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7b6b913b-28d2-421f-84de-d0636017bfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:59.694147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b913b-28d2-421f-84de-d0636017bfbf None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:05:59.766797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c720b-1814-430c-bad5-b0997573d740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:05:59.767412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c720b-1814-430c-bad5-b0997573d740 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:05:59.767681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c720b-1814-430c-bad5-b0997573d740 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:05:59.767944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c720b-1814-430c-bad5-b0997573d740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:05:59.769881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8c0c720b-1814-430c-bad5-b0997573d740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:05:59.809337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:05:59.898243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c720b-1814-430c-bad5-b0997573d740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:05:59.899682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c720b-1814-430c-bad5-b0997573d740 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:05:59.920366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 658/1334] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:01.995034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2a97-7091-4bb9-978f-3ed704a32fbe None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:01.995534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2a97-7091-4bb9-978f-3ed704a32fbe None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:01.995534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf2a97-7091-4bb9-978f-3ed704a32fbe None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:01.996990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 677/1335] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:02.002180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29c665-9daf-4b81-845e-ac51a9506c79 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:02.002573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29c665-9daf-4b81-845e-ac51a9506c79 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:02.002738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29c665-9daf-4b81-845e-ac51a9506c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:02.072400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5e29c665-9daf-4b81-845e-ac51a9506c79 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:02.073631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5e29c665-9daf-4b81-845e-ac51a9506c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:02.167282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:02.190391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 659/1336] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:02.232646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2a97-7091-4bb9-978f-3ed704a32fbe None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:02.234171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7faf2a97-7091-4bb9-978f-3ed704a32fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:02.337871 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf2a97-7091-4bb9-978f-3ed704a32fbe None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:02.366154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:02.411964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7478d0b-a264-4477-8e4c-87627c1b8124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:02.412587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478d0b-a264-4477-8e4c-87627c1b8124 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:02.412864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478d0b-a264-4477-8e4c-87627c1b8124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:02.413132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478d0b-a264-4477-8e4c-87627c1b8124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:02.414727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b7478d0b-a264-4477-8e4c-87627c1b8124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:02.533801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7478d0b-a264-4477-8e4c-87627c1b8124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:02.536024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7478d0b-a264-4477-8e4c-87627c1b8124 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:02.553915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 678/1337] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:04.926902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc88c-ebfe-4354-80ff-25f0faf09ada None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:04.927469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc88c-ebfe-4354-80ff-25f0faf09ada None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:04.927469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bc88c-ebfe-4354-80ff-25f0faf09ada None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:04.928798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 660/1338] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:04.934168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d801efb-f7d6-4fa7-9196-590dca60e3ae None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:04.934554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d801efb-f7d6-4fa7-9196-590dca60e3ae None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:04.934715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d801efb-f7d6-4fa7-9196-590dca60e3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:04.999953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0d801efb-f7d6-4fa7-9196-590dca60e3ae None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:05.001173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0d801efb-f7d6-4fa7-9196-590dca60e3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:05.011166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:05.103618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 679/1339] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:06:05.145673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bc88c-ebfe-4354-80ff-25f0faf09ada None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:05.147241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6c4bc88c-ebfe-4354-80ff-25f0faf09ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:05.250631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bc88c-ebfe-4354-80ff-25f0faf09ada None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:05.322661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f9584-d4a7-480c-a1ba-9cf7b9be74f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:05.323252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9584-d4a7-480c-a1ba-9cf7b9be74f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:05.323531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9584-d4a7-480c-a1ba-9cf7b9be74f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:05.323809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9584-d4a7-480c-a1ba-9cf7b9be74f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:05.325428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0e0f9584-d4a7-480c-a1ba-9cf7b9be74f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:05.359614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:05.434938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f9584-d4a7-480c-a1ba-9cf7b9be74f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:05.436375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f9584-d4a7-480c-a1ba-9cf7b9be74f6 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:05.454080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 661/1340] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:09.489143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640db06-198c-4440-bc88-dedff69a760a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:09.489685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640db06-198c-4440-bc88-dedff69a760a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:09.489685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640db06-198c-4440-bc88-dedff69a760a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:09.491020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 680/1341] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:09.496395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9770-b4e8-4336-ac81-3fe2702121c9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:09.496887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9770-b4e8-4336-ac81-3fe2702121c9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:09.496994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9770-b4e8-4336-ac81-3fe2702121c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:09.565071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9e3d9770-b4e8-4336-ac81-3fe2702121c9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:09.566695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9e3d9770-b4e8-4336-ac81-3fe2702121c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:09.652007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:09.664288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 662/1342] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:09.703832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640db06-198c-4440-bc88-dedff69a760a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:09.705424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e640db06-198c-4440-bc88-dedff69a760a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:09.811734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640db06-198c-4440-bc88-dedff69a760a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:09.849479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:09.886907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a5691-de37-4990-9f67-17ae39c0a403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:09.887183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a5691-de37-4990-9f67-17ae39c0a403 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:09.887608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a5691-de37-4990-9f67-17ae39c0a403 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:09.887886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a5691-de37-4990-9f67-17ae39c0a403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:09.889546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f34a5691-de37-4990-9f67-17ae39c0a403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:10.006902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a5691-de37-4990-9f67-17ae39c0a403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:10.008373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a5691-de37-4990-9f67-17ae39c0a403 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:10.026484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 681/1343] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:06:12.789620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92ea6-b53a-4f7b-a687-a7230d9f6ad2 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:12.790228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92ea6-b53a-4f7b-a687-a7230d9f6ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:12.790228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92ea6-b53a-4f7b-a687-a7230d9f6ad2 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:12.791506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 663/1344] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:12.796792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa119e-9e41-4593-9bc2-dd09d783c467 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:12.797236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa119e-9e41-4593-9bc2-dd09d783c467 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:12.797385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa119e-9e41-4593-9bc2-dd09d783c467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:12.866473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a0aa119e-9e41-4593-9bc2-dd09d783c467 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:12.867659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a0aa119e-9e41-4593-9bc2-dd09d783c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:12.885998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:12.967508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 682/1345] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:13.005378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92ea6-b53a-4f7b-a687-a7230d9f6ad2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:13.006930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e3e92ea6-b53a-4f7b-a687-a7230d9f6ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:13.122068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92ea6-b53a-4f7b-a687-a7230d9f6ad2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:13.207527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844651e-c4d2-4a38-b13c-cfd27f5b92c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:13.208121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844651e-c4d2-4a38-b13c-cfd27f5b92c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:13.208398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844651e-c4d2-4a38-b13c-cfd27f5b92c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:13.208704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844651e-c4d2-4a38-b13c-cfd27f5b92c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:13.210331 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8844651e-c4d2-4a38-b13c-cfd27f5b92c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:13.254563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:13.321445 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844651e-c4d2-4a38-b13c-cfd27f5b92c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:13.323559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844651e-c4d2-4a38-b13c-cfd27f5b92c0 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:13.343453 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 664/1346] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:17.430405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd07c49-f52d-4067-ae71-13040b07e100 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:17.430964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd07c49-f52d-4067-ae71-13040b07e100 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:17.430964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd07c49-f52d-4067-ae71-13040b07e100 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:17.432552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 683/1347] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:17.437918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9637ec9-14e3-4240-8dc2-7fa4a0412c73 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:17.438297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9637ec9-14e3-4240-8dc2-7fa4a0412c73 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:17.438459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9637ec9-14e3-4240-8dc2-7fa4a0412c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:17.511643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f9637ec9-14e3-4240-8dc2-7fa4a0412c73 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:17.512901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f9637ec9-14e3-4240-8dc2-7fa4a0412c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:17.617657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:17.621363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 665/1348] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:17.670927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd07c49-f52d-4067-ae71-13040b07e100 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:17.672753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7cd07c49-f52d-4067-ae71-13040b07e100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:17.809147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd07c49-f52d-4067-ae71-13040b07e100 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:17.872567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:17.909511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130793a-f079-40c2-93ae-bbf09d6f1846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:17.910247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130793a-f079-40c2-93ae-bbf09d6f1846 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:17.910526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130793a-f079-40c2-93ae-bbf09d6f1846 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:17.910794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130793a-f079-40c2-93ae-bbf09d6f1846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:17.912494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d130793a-f079-40c2-93ae-bbf09d6f1846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:18.047653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130793a-f079-40c2-93ae-bbf09d6f1846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:18.049144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130793a-f079-40c2-93ae-bbf09d6f1846 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:18.072088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 684/1349] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:23.059820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd592e-b4e6-4724-b024-0271fe4dae73 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:23.060661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd592e-b4e6-4724-b024-0271fe4dae73 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:23.061108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd592e-b4e6-4724-b024-0271fe4dae73 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:23.062847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 666/1350] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:23.070558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b790-6544-4b96-8c1f-b80629b9d087 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:23.071021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b790-6544-4b96-8c1f-b80629b9d087 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:23.071100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571b790-6544-4b96-8c1f-b80629b9d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:23.174522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8571b790-6544-4b96-8c1f-b80629b9d087 None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:23.176421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8571b790-6544-4b96-8c1f-b80629b9d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:23.228099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:23.350927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 685/1351] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:06:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:27.473587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600ec13-63a0-4de8-bd52-29658be29a38 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:27.474144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600ec13-63a0-4de8-bd52-29658be29a38 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:27.474144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600ec13-63a0-4de8-bd52-29658be29a38 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:27.475710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 667/1352] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:27.482001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4f79b-f41f-4947-ac84-0c5fb91a1b1a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:27.482462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4f79b-f41f-4947-ac84-0c5fb91a1b1a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:27.482554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4f79b-f41f-4947-ac84-0c5fb91a1b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:27.569513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2af4f79b-f41f-4947-ac84-0c5fb91a1b1a None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:27.570714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2af4f79b-f41f-4947-ac84-0c5fb91a1b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:27.648118 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:27.688085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 686/1353] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:06:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:29.110018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ee29-17ae-4372-a074-2c4819734471 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:29.110520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ee29-17ae-4372-a074-2c4819734471 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:29.110618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4ee29-17ae-4372-a074-2c4819734471 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:29.111872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 668/1354] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:29.117210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28e12-be12-4904-b0af-96f85e2e6fe5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:29.117658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28e12-be12-4904-b0af-96f85e2e6fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:29.117892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28e12-be12-4904-b0af-96f85e2e6fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:29.204351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7ae28e12-be12-4904-b0af-96f85e2e6fe5 None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:29.205906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ae28e12-be12-4904-b0af-96f85e2e6fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:29.313739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 687/1355] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:06:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:32.815808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad902c40-0b0e-4d50-8a0d-dfc7e4f379fb None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:32.816899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad902c40-0b0e-4d50-8a0d-dfc7e4f379fb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:32.817388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad902c40-0b0e-4d50-8a0d-dfc7e4f379fb None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:32.820831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 669/1356] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:32.825507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317890-2c7d-47c3-b0c6-9424e5251b09 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:32.825962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317890-2c7d-47c3-b0c6-9424e5251b09 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:32.826549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41317890-2c7d-47c3-b0c6-9424e5251b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:32.873048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:32.915927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-41317890-2c7d-47c3-b0c6-9424e5251b09 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:32.917786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-41317890-2c7d-47c3-b0c6-9424e5251b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:33.025551 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 688/1357] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:06:33.083685 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e191c-77c6-4dea-8ca4-1dad83e73b4a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:33.084102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e191c-77c6-4dea-8ca4-1dad83e73b4a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:33.084617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e191c-77c6-4dea-8ca4-1dad83e73b4a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:33.086274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 670/1358] 10.222.0.70 () {54 vars in 962 bytes} [Tue Aug 9 19:06:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:33.096794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a46a2a-18fc-406f-aff0-bf193e321548 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:33.097329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a46a2a-18fc-406f-aff0-bf193e321548 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:33.097788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a46a2a-18fc-406f-aff0-bf193e321548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:33.186114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a7a46a2a-18fc-406f-aff0-bf193e321548 None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:33.187716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a7a46a2a-18fc-406f-aff0-bf193e321548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:33.247137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:33.326711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 689/1359] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Aug 9 19:06:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:06:33.335893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e191c-77c6-4dea-8ca4-1dad83e73b4a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:33.337635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5c6e191c-77c6-4dea-8ca4-1dad83e73b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:33.507388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e191c-77c6-4dea-8ca4-1dad83e73b4a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:33.588696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:33.632555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c16b2-6363-46ed-adeb-faa8d29e3e2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:33.633170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c16b2-6363-46ed-adeb-faa8d29e3e2d service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:33.633450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c16b2-6363-46ed-adeb-faa8d29e3e2d service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:33.633723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c16b2-6363-46ed-adeb-faa8d29e3e2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:33.635500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8f6c16b2-6363-46ed-adeb-faa8d29e3e2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:33.731810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c16b2-6363-46ed-adeb-faa8d29e3e2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:33.733706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c16b2-6363-46ed-adeb-faa8d29e3e2d service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:33.752852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 671/1360] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:06:36.016457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4e6c-98a0-43b5-bda1-a44569ee9cf0 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:36.016971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4e6c-98a0-43b5-bda1-a44569ee9cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:36.016971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4e6c-98a0-43b5-bda1-a44569ee9cf0 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:36.018366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 690/1361] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:36.023937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40975fac-f2a6-4ee1-a992-b80aa734635f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:36.024328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40975fac-f2a6-4ee1-a992-b80aa734635f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:36.024542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40975fac-f2a6-4ee1-a992-b80aa734635f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:36.098239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-40975fac-f2a6-4ee1-a992-b80aa734635f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:36.099484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-40975fac-f2a6-4ee1-a992-b80aa734635f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:36.142889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:36.209738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 672/1362] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:36.253749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b4e6c-98a0-43b5-bda1-a44569ee9cf0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:36.255496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d29b4e6c-98a0-43b5-bda1-a44569ee9cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:36.460598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b4e6c-98a0-43b5-bda1-a44569ee9cf0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:36.470765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:36.616366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501455cf-1dc6-48bc-9016-64397c51ca03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:36.617151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501455cf-1dc6-48bc-9016-64397c51ca03 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:36.617537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501455cf-1dc6-48bc-9016-64397c51ca03 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:36.617910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501455cf-1dc6-48bc-9016-64397c51ca03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:36.619597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-501455cf-1dc6-48bc-9016-64397c51ca03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:36.721422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501455cf-1dc6-48bc-9016-64397c51ca03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:36.724128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501455cf-1dc6-48bc-9016-64397c51ca03 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:36.744809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 691/1363] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:39.460298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c3170-edcd-4a16-9174-111adfa7a8fd None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:39.460864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c3170-edcd-4a16-9174-111adfa7a8fd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:39.460864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c3170-edcd-4a16-9174-111adfa7a8fd None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:39.462448 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 673/1364] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:39.467755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4fa99-4359-4103-adbe-c2839631f7d2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:39.468239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4fa99-4359-4103-adbe-c2839631f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:39.468332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4fa99-4359-4103-adbe-c2839631f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:39.484411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:39.542928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-14d4fa99-4359-4103-adbe-c2839631f7d2 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:39.543867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-14d4fa99-4359-4103-adbe-c2839631f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:39.652550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 692/1365] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:06:39.697244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3f58-8611-46d8-84df-15595ee31d91 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:39.697534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3f58-8611-46d8-84df-15595ee31d91 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:39.698077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e3f58-8611-46d8-84df-15595ee31d91 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:39.699491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 674/1366] 10.222.0.70 () {54 vars in 962 bytes} [Tue Aug 9 19:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:39.704884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27062f-1472-462f-bd4b-afd3cbebddee None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:39.705171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27062f-1472-462f-bd4b-afd3cbebddee None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:39.705443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27062f-1472-462f-bd4b-afd3cbebddee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:39.787945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8e27062f-1472-462f-bd4b-afd3cbebddee None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:39.789199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8e27062f-1472-462f-bd4b-afd3cbebddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:39.828929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:39.918986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 693/1367] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Aug 9 19:06:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:39.928091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e3f58-8611-46d8-84df-15595ee31d91 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:39.929838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d19e3f58-8611-46d8-84df-15595ee31d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:40.099433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e3f58-8611-46d8-84df-15595ee31d91 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:40.123241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:40.229573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874695e3-6290-42c7-a290-504a4df769dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:40.230633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874695e3-6290-42c7-a290-504a4df769dd service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:40.231114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874695e3-6290-42c7-a290-504a4df769dd service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:40.231709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874695e3-6290-42c7-a290-504a4df769dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:40.233565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-874695e3-6290-42c7-a290-504a4df769dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:40.338967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874695e3-6290-42c7-a290-504a4df769dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:40.340634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874695e3-6290-42c7-a290-504a4df769dd service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:40.360944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 675/1368] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:06:41.229163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae692ad7-58c7-4bfb-ab13-f601299872d7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:41.229768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae692ad7-58c7-4bfb-ab13-f601299872d7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:41.229768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae692ad7-58c7-4bfb-ab13-f601299872d7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:41.231287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 694/1369] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:41.236342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d5c9-45cb-451c-b6c9-21eaf8141a06 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:41.236842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d5c9-45cb-451c-b6c9-21eaf8141a06 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:41.236924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d5c9-45cb-451c-b6c9-21eaf8141a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:41.270543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:41.308799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7e47d5c9-45cb-451c-b6c9-21eaf8141a06 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:41.309975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7e47d5c9-45cb-451c-b6c9-21eaf8141a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:41.419667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 676/1370] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:41.766768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef028a-b9b5-4370-86c5-0f849314ca2e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:41.766768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef028a-b9b5-4370-86c5-0f849314ca2e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:41.766768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef028a-b9b5-4370-86c5-0f849314ca2e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:41.767887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 695/1371] 10.222.0.70 () {54 vars in 964 bytes} [Tue Aug 9 19:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:41.773650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d43d5-f5a4-4e26-ae0d-06dff1002f4b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:41.774002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d43d5-f5a4-4e26-ae0d-06dff1002f4b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:41.774343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d43d5-f5a4-4e26-ae0d-06dff1002f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:41.856239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c65d43d5-f5a4-4e26-ae0d-06dff1002f4b None None] MFA Rules not processed for user `00be9b6d551a4ba39718a44101ecf9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:41.857551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c65d43d5-f5a4-4e26-ae0d-06dff1002f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:41.906688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:41.938290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 677/1372] 10.222.0.70 () {58 vars in 1061 bytes} [Tue Aug 9 19:06:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:41.946446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef028a-b9b5-4370-86c5-0f849314ca2e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:41.948049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eeef028a-b9b5-4370-86c5-0f849314ca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:42.067084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef028a-b9b5-4370-86c5-0f849314ca2e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:42.115654 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:42.158297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da96359-9615-45b4-9f1a-e3c415d22240 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:42.159149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96359-9615-45b4-9f1a-e3c415d22240 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:42.160096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96359-9615-45b4-9f1a-e3c415d22240 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:42.160496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96359-9615-45b4-9f1a-e3c415d22240 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:42.162442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9da96359-9615-45b4-9f1a-e3c415d22240 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:42.282312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da96359-9615-45b4-9f1a-e3c415d22240 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:42.283826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da96359-9615-45b4-9f1a-e3c415d22240 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:42.303292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 696/1373] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:42.468992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7978500-a7cc-4533-80e6-6c1d89ad8816 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:42.469698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7978500-a7cc-4533-80e6-6c1d89ad8816 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:42.469698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7978500-a7cc-4533-80e6-6c1d89ad8816 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:42.471088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 678/1374] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:42.476467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51111bf-dca3-4d6a-868d-a87f7a00d563 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:42.476993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51111bf-dca3-4d6a-868d-a87f7a00d563 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:42.476993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51111bf-dca3-4d6a-868d-a87f7a00d563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:42.546263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a51111bf-dca3-4d6a-868d-a87f7a00d563 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:42.547331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a51111bf-dca3-4d6a-868d-a87f7a00d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:42.570656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:42.652586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 697/1375] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:42.697766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7978500-a7cc-4533-80e6-6c1d89ad8816 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:42.701090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f7978500-a7cc-4533-80e6-6c1d89ad8816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:42.852286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7978500-a7cc-4533-80e6-6c1d89ad8816 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:42.904253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:42.967398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659e962-c9f4-4fbd-b80d-e483d73b7b33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:42.968000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659e962-c9f4-4fbd-b80d-e483d73b7b33 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:42.968281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659e962-c9f4-4fbd-b80d-e483d73b7b33 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:42.968583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659e962-c9f4-4fbd-b80d-e483d73b7b33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:42.970193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9659e962-c9f4-4fbd-b80d-e483d73b7b33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:43.064147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659e962-c9f4-4fbd-b80d-e483d73b7b33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:43.066582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659e962-c9f4-4fbd-b80d-e483d73b7b33 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:43.084484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 679/1376] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:45.171057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117169e-fd82-4cdd-8e4d-f7b40b25a414 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:45.171597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117169e-fd82-4cdd-8e4d-f7b40b25a414 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:45.171597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117169e-fd82-4cdd-8e4d-f7b40b25a414 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:45.173008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 698/1377] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:45.178940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586cd93-033e-4a4e-9c56-0fed88922614 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:45.179487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586cd93-033e-4a4e-9c56-0fed88922614 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:45.179487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586cd93-033e-4a4e-9c56-0fed88922614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:45.256792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5586cd93-033e-4a4e-9c56-0fed88922614 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:45.257989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5586cd93-033e-4a4e-9c56-0fed88922614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:45.264625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:45.362333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 680/1378] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:45.408328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117169e-fd82-4cdd-8e4d-f7b40b25a414 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:45.410598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1117169e-fd82-4cdd-8e4d-f7b40b25a414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:45.578820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117169e-fd82-4cdd-8e4d-f7b40b25a414 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:45.585090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:45.698748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92bb73-c0cc-4c93-a872-5e51690a8637 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:45.699352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92bb73-c0cc-4c93-a872-5e51690a8637 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:45.699626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92bb73-c0cc-4c93-a872-5e51690a8637 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:45.699889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92bb73-c0cc-4c93-a872-5e51690a8637 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:45.701503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7f92bb73-c0cc-4c93-a872-5e51690a8637 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:45.805016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92bb73-c0cc-4c93-a872-5e51690a8637 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:45.806923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92bb73-c0cc-4c93-a872-5e51690a8637 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:45.825698 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 699/1379] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:47.925151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf8f1-acba-4064-8571-9f890b6de4bd None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:47.925730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf8f1-acba-4064-8571-9f890b6de4bd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:47.925828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf8f1-acba-4064-8571-9f890b6de4bd None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:47.927647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 681/1380] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:47.933724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4824d-022e-4e2c-b0d3-4bff43e4d3bf None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:47.934244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4824d-022e-4e2c-b0d3-4bff43e4d3bf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:47.934668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4824d-022e-4e2c-b0d3-4bff43e4d3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:47.960146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:48.024526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f9f4824d-022e-4e2c-b0d3-4bff43e4d3bf None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:48.025648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f9f4824d-022e-4e2c-b0d3-4bff43e4d3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:48.129253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 700/1381] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:48.172672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbf8f1-acba-4064-8571-9f890b6de4bd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:48.174180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-03cbf8f1-acba-4064-8571-9f890b6de4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:48.338443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:48.350310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbf8f1-acba-4064-8571-9f890b6de4bd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:48.473243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3c408-d22c-456d-b700-923b2edcd9f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:48.473939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3c408-d22c-456d-b700-923b2edcd9f0 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:48.474142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3c408-d22c-456d-b700-923b2edcd9f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:48.474391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3c408-d22c-456d-b700-923b2edcd9f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:48.475985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-67b3c408-d22c-456d-b700-923b2edcd9f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:48.570856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3c408-d22c-456d-b700-923b2edcd9f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:48.572357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3c408-d22c-456d-b700-923b2edcd9f0 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:48.584073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:48.592930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 682/1382] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:50.734640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fdd7a-3462-4419-ad10-f70ed11cda36 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:50.735226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fdd7a-3462-4419-ad10-f70ed11cda36 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:50.735226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fdd7a-3462-4419-ad10-f70ed11cda36 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:50.736558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 701/1383] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:06:50.742474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96b57-cf58-48ac-8ea2-d138f8671266 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:50.742931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96b57-cf58-48ac-8ea2-d138f8671266 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:50.743056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96b57-cf58-48ac-8ea2-d138f8671266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:50.814826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1ae96b57-cf58-48ac-8ea2-d138f8671266 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:50.816136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1ae96b57-cf58-48ac-8ea2-d138f8671266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:50.921780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 683/1384] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:50.966933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fdd7a-3462-4419-ad10-f70ed11cda36 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:50.968546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1b3fdd7a-3462-4419-ad10-f70ed11cda36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:51.057518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:51.124765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fdd7a-3462-4419-ad10-f70ed11cda36 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:51.243270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14d1c2-730e-4bf3-937f-c83534cc203e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:51.244078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14d1c2-730e-4bf3-937f-c83534cc203e service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:51.244361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14d1c2-730e-4bf3-937f-c83534cc203e service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:51.244670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14d1c2-730e-4bf3-937f-c83534cc203e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:51.246327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ed14d1c2-730e-4bf3-937f-c83534cc203e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:51.298227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:51.348712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14d1c2-730e-4bf3-937f-c83534cc203e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:51.350657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14d1c2-730e-4bf3-937f-c83534cc203e service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:51.377742 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 702/1385] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:53.475474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641c5a-8f45-4a99-bb5e-0bf250ba441c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:53.476050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641c5a-8f45-4a99-bb5e-0bf250ba441c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:53.476050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641c5a-8f45-4a99-bb5e-0bf250ba441c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:53.477614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 684/1386] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:53.483192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d484e1-0b7f-4708-87c5-2da10343a5f6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:53.483639 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d484e1-0b7f-4708-87c5-2da10343a5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:53.483753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d484e1-0b7f-4708-87c5-2da10343a5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:53.559420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-26d484e1-0b7f-4708-87c5-2da10343a5f6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:53.561142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-26d484e1-0b7f-4708-87c5-2da10343a5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:53.671632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 703/1387] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:53.718809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31641c5a-8f45-4a99-bb5e-0bf250ba441c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:53.720348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-31641c5a-8f45-4a99-bb5e-0bf250ba441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:53.808236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:53.886312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31641c5a-8f45-4a99-bb5e-0bf250ba441c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:54.025583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c01282-0c69-4b87-a241-a5a90aef6770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:54.026137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01282-0c69-4b87-a241-a5a90aef6770 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:54.026414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01282-0c69-4b87-a241-a5a90aef6770 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:54.026706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01282-0c69-4b87-a241-a5a90aef6770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:54.028375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c3c01282-0c69-4b87-a241-a5a90aef6770 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:54.069239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:54.128067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c01282-0c69-4b87-a241-a5a90aef6770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:54.130285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c01282-0c69-4b87-a241-a5a90aef6770 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:54.154311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 685/1388] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:56.378333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103a0f-241d-4cca-a557-eeed33a1a5f7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:56.379090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103a0f-241d-4cca-a557-eeed33a1a5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:56.379090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103a0f-241d-4cca-a557-eeed33a1a5f7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:56.380278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 704/1389] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:56.385750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4953291-0f4f-4315-b004-459b754ffb26 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:56.386213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4953291-0f4f-4315-b004-459b754ffb26 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:56.386294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4953291-0f4f-4315-b004-459b754ffb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:56.453712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a4953291-0f4f-4315-b004-459b754ffb26 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:56.454835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a4953291-0f4f-4315-b004-459b754ffb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:56.552340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 686/1390] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:56.597892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103a0f-241d-4cca-a557-eeed33a1a5f7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:56.599407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d4103a0f-241d-4cca-a557-eeed33a1a5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:56.633519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:56.749244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103a0f-241d-4cca-a557-eeed33a1a5f7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:56.863642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:56.875790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d54d1-4070-4239-a9d8-adef81fe1943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:56.876394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d54d1-4070-4239-a9d8-adef81fe1943 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:56.876701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d54d1-4070-4239-a9d8-adef81fe1943 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:56.876971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d54d1-4070-4239-a9d8-adef81fe1943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:56.878876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c82d54d1-4070-4239-a9d8-adef81fe1943 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:56.968734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d54d1-4070-4239-a9d8-adef81fe1943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:56.970335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d54d1-4070-4239-a9d8-adef81fe1943 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:56.989184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 705/1391] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:57.534550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc11a-57b1-4c27-af0b-564f581a9cf5 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:57.535346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc11a-57b1-4c27-af0b-564f581a9cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:57.535346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc11a-57b1-4c27-af0b-564f581a9cf5 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:57.536850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 687/1392] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:57.541812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba737b-cea3-4bc4-a573-a2cd79ac414d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:57.542253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba737b-cea3-4bc4-a573-a2cd79ac414d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:57.542393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba737b-cea3-4bc4-a573-a2cd79ac414d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:57.609615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-52ba737b-cea3-4bc4-a573-a2cd79ac414d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:57.610813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-52ba737b-cea3-4bc4-a573-a2cd79ac414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:57.668803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:57.708521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 706/1393] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:57.764301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1db21-f94e-48e6-a07e-34851d382d64 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:57.764620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1db21-f94e-48e6-a07e-34851d382d64 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:57.764900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1db21-f94e-48e6-a07e-34851d382d64 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:57.766350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 688/1394] 10.222.0.70 () {54 vars in 967 bytes} [Tue Aug 9 19:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:06:57.771716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943ac8b-4a58-43be-90c7-1ef4b5736e4d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:57.772008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943ac8b-4a58-43be-90c7-1ef4b5736e4d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:57.772283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943ac8b-4a58-43be-90c7-1ef4b5736e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:57.847253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f943ac8b-4a58-43be-90c7-1ef4b5736e4d None None] MFA Rules not processed for user `92a88817f3a4402290e9358826a63904`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:57.848421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f943ac8b-4a58-43be-90c7-1ef4b5736e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:57.923260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 707/1395] 10.222.0.70 () {58 vars in 1064 bytes} [Tue Aug 9 19:06:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:57.930961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1db21-f94e-48e6-a07e-34851d382d64 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:57.932515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-63c1db21-f94e-48e6-a07e-34851d382d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:57.945834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:58.040678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1db21-f94e-48e6-a07e-34851d382d64 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:58.117086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62ba63-c0df-481b-9cdf-681c362f3d62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a88817f3a4402290e9358826a63904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:58.117699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ba63-c0df-481b-9cdf-681c362f3d62 service glance] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:58.117977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ba63-c0df-481b-9cdf-681c362f3d62 service glance] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:58.118249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ba63-c0df-481b-9cdf-681c362f3d62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:58.119902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1e62ba63-c0df-481b-9cdf-681c362f3d62 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:58.193090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:58.235865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62ba63-c0df-481b-9cdf-681c362f3d62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:58.238125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62ba63-c0df-481b-9cdf-681c362f3d62 service glance] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:58.257772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 689/1396] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:06:58.993580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b0c6-72b4-4af2-a314-e49243378a45 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:58.994087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b0c6-72b4-4af2-a314-e49243378a45 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:58.994197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b0c6-72b4-4af2-a314-e49243378a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:59.061314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-aed7b0c6-72b4-4af2-a314-e49243378a45 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:59.062509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aed7b0c6-72b4-4af2-a314-e49243378a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:59.070875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:59.097160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc47756-8064-4f0f-8576-593a675b3b9c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:59.097767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc47756-8064-4f0f-8576-593a675b3b9c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:59.097981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc47756-8064-4f0f-8576-593a675b3b9c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:59.099411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 690/1397] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:06:59.104718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629de35-6240-4120-93d6-54b6ffc554e9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:59.104996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629de35-6240-4120-93d6-54b6ffc554e9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:59.105309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629de35-6240-4120-93d6-54b6ffc554e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:59.158734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 708/1398] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:06:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:06:59.177444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a629de35-6240-4120-93d6-54b6ffc554e9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:59.178679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a629de35-6240-4120-93d6-54b6ffc554e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:59.203505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb72dff-8305-44d1-8a82-b33e80f2fcc4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:59.203789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb72dff-8305-44d1-8a82-b33e80f2fcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:59.204060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb72dff-8305-44d1-8a82-b33e80f2fcc4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:59.205536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 709/1399] 10.222.0.70 () {54 vars in 943 bytes} [Tue Aug 9 19:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:06:59.211880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9c1a2-d4c4-4fe9-be60-82f279c982cb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:59.212233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9c1a2-d4c4-4fe9-be60-82f279c982cb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:59.212617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9c1a2-d4c4-4fe9-be60-82f279c982cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:59.284031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 691/1400] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:06:59.293764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-90f9c1a2-d4c4-4fe9-be60-82f279c982cb None None] MFA Rules not processed for user `141d2c2f92c842feb6155aae1e9edfd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:06:59.294983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-90f9c1a2-d4c4-4fe9-be60-82f279c982cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:59.332515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629de35-6240-4120-93d6-54b6ffc554e9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:59.333751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a629de35-6240-4120-93d6-54b6ffc554e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:59.363191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:59.382922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:59.412070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 710/1401] 10.222.0.70 () {58 vars in 1040 bytes} [Tue Aug 9 19:06:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:06:59.422594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9c1a2-d4c4-4fe9-be60-82f279c982cb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:06:59.432610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9c1a2-d4c4-4fe9-be60-82f279c982cb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:59.459641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629de35-6240-4120-93d6-54b6ffc554e9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:06:59.544007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0a156-0693-4edb-8031-5778bfc54da0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:59.544646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0a156-0693-4edb-8031-5778bfc54da0 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:59.544924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0a156-0693-4edb-8031-5778bfc54da0 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:59.545194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0a156-0693-4edb-8031-5778bfc54da0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:59.546868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d8f0a156-0693-4edb-8031-5778bfc54da0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:59.580614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892bbd1-e017-4715-a488-15b7cb556b34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:06:59.582161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892bbd1-e017-4715-a488-15b7cb556b34 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:06:59.582438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892bbd1-e017-4715-a488-15b7cb556b34 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:06:59.582708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892bbd1-e017-4715-a488-15b7cb556b34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:06:59.584705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7892bbd1-e017-4715-a488-15b7cb556b34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:06:59.602955 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:06:59.632188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0a156-0693-4edb-8031-5778bfc54da0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:59.633825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0a156-0693-4edb-8031-5778bfc54da0 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:59.641799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 711/1402] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 19:06:59.678993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892bbd1-e017-4715-a488-15b7cb556b34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:06:59.680992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892bbd1-e017-4715-a488-15b7cb556b34 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:06:59.703718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 692/1403] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:07:01.844360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2705-7739-4ab0-bb5f-ad59fbf52b5d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:01.844964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2705-7739-4ab0-bb5f-ad59fbf52b5d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:01.844964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2705-7739-4ab0-bb5f-ad59fbf52b5d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:01.847047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 712/1404] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:01.852116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd2dae-2db6-4cfd-9463-de623c101d66 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:01.852627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd2dae-2db6-4cfd-9463-de623c101d66 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:01.852724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd2dae-2db6-4cfd-9463-de623c101d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:01.957458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bbdd2dae-2db6-4cfd-9463-de623c101d66 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:01.957458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bbdd2dae-2db6-4cfd-9463-de623c101d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:02.058776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:02.067879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 693/1405] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:02.120692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2705-7739-4ab0-bb5f-ad59fbf52b5d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:02.122691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-861e2705-7739-4ab0-bb5f-ad59fbf52b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:02.133317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:02.279439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2705-7739-4ab0-bb5f-ad59fbf52b5d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:02.375381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:02.413275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de6169-cb76-456b-9b2c-36461a9226c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:02.413880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de6169-cb76-456b-9b2c-36461a9226c4 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:02.414151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de6169-cb76-456b-9b2c-36461a9226c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:02.414413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de6169-cb76-456b-9b2c-36461a9226c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:02.416046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-10de6169-cb76-456b-9b2c-36461a9226c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:02.509969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de6169-cb76-456b-9b2c-36461a9226c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:02.511574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de6169-cb76-456b-9b2c-36461a9226c4 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:02.532074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 713/1406] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:04.639360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061c8f-99c5-416c-b828-caad49753b73 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:04.639920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061c8f-99c5-416c-b828-caad49753b73 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:04.639920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061c8f-99c5-416c-b828-caad49753b73 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:04.641360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 694/1407] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:04.647232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5cf6-9c52-4106-817b-018b77c90ff0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:04.647662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5cf6-9c52-4106-817b-018b77c90ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:04.647850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5cf6-9c52-4106-817b-018b77c90ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:04.727057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-72eb5cf6-9c52-4106-817b-018b77c90ff0 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:04.728315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-72eb5cf6-9c52-4106-817b-018b77c90ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:04.791704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:04.862814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 714/1408] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:07:04.907911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061c8f-99c5-416c-b828-caad49753b73 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:04.910044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ac061c8f-99c5-416c-b828-caad49753b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:05.068025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061c8f-99c5-416c-b828-caad49753b73 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:05.144894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:05.198244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc6e25-88ac-43c2-8cd4-39047a4d2409 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:05.198855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc6e25-88ac-43c2-8cd4-39047a4d2409 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:05.199132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc6e25-88ac-43c2-8cd4-39047a4d2409 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:05.199402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc6e25-88ac-43c2-8cd4-39047a4d2409 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:05.201050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-42fc6e25-88ac-43c2-8cd4-39047a4d2409 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:05.304593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc6e25-88ac-43c2-8cd4-39047a4d2409 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:05.307042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc6e25-88ac-43c2-8cd4-39047a4d2409 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:05.326277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 695/1409] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:14.860231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=23 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:15.323771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=64 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:22.472400 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b9b43-bb5a-4d0b-89c6-7442a5329711 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:22.473162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b9b43-bb5a-4d0b-89c6-7442a5329711 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:22.473162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b9b43-bb5a-4d0b-89c6-7442a5329711 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:22.474428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 715/1410] 10.222.0.70 () {54 vars in 918 bytes} [Tue Aug 9 19:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:22.479868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26dbfa-cc57-4590-b771-d8d263f8a5e9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:22.480318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26dbfa-cc57-4590-b771-d8d263f8a5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:22.480414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26dbfa-cc57-4590-b771-d8d263f8a5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:22.557682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-cc26dbfa-cc57-4590-b771-d8d263f8a5e9 None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:22.558888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cc26dbfa-cc57-4590-b771-d8d263f8a5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:22.662658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 696/1411] 10.222.0.70 () {58 vars in 1015 bytes} [Tue Aug 9 19:07:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:22.677472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277371-768a-4f3a-90d9-05849bfe5b06 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:22.677749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277371-768a-4f3a-90d9-05849bfe5b06 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:22.678017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277371-768a-4f3a-90d9-05849bfe5b06 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:22.679331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 716/1412] 10.222.0.70 () {54 vars in 918 bytes} [Tue Aug 9 19:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:22.684153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0685dd7-0df3-42af-aa60-fde243aa2b6e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:22.684466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0685dd7-0df3-42af-aa60-fde243aa2b6e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:22.684750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0685dd7-0df3-42af-aa60-fde243aa2b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:22.758139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:22.760770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b0685dd7-0df3-42af-aa60-fde243aa2b6e None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:22.762555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b0685dd7-0df3-42af-aa60-fde243aa2b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:22.881148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 697/1413] 10.222.0.70 () {58 vars in 1015 bytes} [Tue Aug 9 19:07:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:25.345661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708de0a-ab70-49cb-b445-98b642506ead None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:25.346200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708de0a-ab70-49cb-b445-98b642506ead None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:25.346331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708de0a-ab70-49cb-b445-98b642506ead None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:25.348378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:25.353720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77c68-fb6f-4ea2-91db-b0c600f02ac7 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:25.354238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77c68-fb6f-4ea2-91db-b0c600f02ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:25.354342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c77c68-fb6f-4ea2-91db-b0c600f02ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:25.427990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-10c77c68-fb6f-4ea2-91db-b0c600f02ac7 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:25.429244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-10c77c68-fb6f-4ea2-91db-b0c600f02ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:25.483708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:25.526464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:25.569270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708de0a-ab70-49cb-b445-98b642506ead None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:25.570791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8708de0a-ab70-49cb-b445-98b642506ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:25.724646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708de0a-ab70-49cb-b445-98b642506ead None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:25.802672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:25.844118 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fdcd8-7f52-4c9a-82a0-1ab7208f879e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:25.844747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fdcd8-7f52-4c9a-82a0-1ab7208f879e service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:25.845025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fdcd8-7f52-4c9a-82a0-1ab7208f879e service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:25.845296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fdcd8-7f52-4c9a-82a0-1ab7208f879e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:25.846973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-000fdcd8-7f52-4c9a-82a0-1ab7208f879e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:25.935426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fdcd8-7f52-4c9a-82a0-1ab7208f879e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:25.937060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fdcd8-7f52-4c9a-82a0-1ab7208f879e service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:25.955046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 718/1416] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:26.860923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a148a-2e2b-43b8-8f17-a739b5d798c4 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:26.861560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a148a-2e2b-43b8-8f17-a739b5d798c4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:26.861560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a148a-2e2b-43b8-8f17-a739b5d798c4 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:26.862958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:07:26.868812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df368-578f-42b6-83ae-dc0b74a1208c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:26.869246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df368-578f-42b6-83ae-dc0b74a1208c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:26.869372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453df368-578f-42b6-83ae-dc0b74a1208c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:26.947352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-453df368-578f-42b6-83ae-dc0b74a1208c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:26.948603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-453df368-578f-42b6-83ae-dc0b74a1208c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:26.974719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:27.044541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:27.195870 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a148a-2e2b-43b8-8f17-a739b5d798c4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:27.197514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7a4a148a-2e2b-43b8-8f17-a739b5d798c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:27.339778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a148a-2e2b-43b8-8f17-a739b5d798c4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:27.381124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:27.456223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbe11f-f220-4b96-8bb2-0078c5f0c0e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:27.456925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe11f-f220-4b96-8bb2-0078c5f0c0e7 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:27.457394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe11f-f220-4b96-8bb2-0078c5f0c0e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:27.457685 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbe11f-f220-4b96-8bb2-0078c5f0c0e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:27.459306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bcfbe11f-f220-4b96-8bb2-0078c5f0c0e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:27.549084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbe11f-f220-4b96-8bb2-0078c5f0c0e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:27.551306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbe11f-f220-4b96-8bb2-0078c5f0c0e7 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:27.569197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 700/1419] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:28.468646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69b6ec-a483-4c26-903a-9435dbe3e91b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:28.469178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69b6ec-a483-4c26-903a-9435dbe3e91b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:28.469178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69b6ec-a483-4c26-903a-9435dbe3e91b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:28.470539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:28.476402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ff623-6373-4f7d-b352-18831d65a3da None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:28.476836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ff623-6373-4f7d-b352-18831d65a3da None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:28.476970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ff623-6373-4f7d-b352-18831d65a3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:28.533307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:28.555882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c73ff623-6373-4f7d-b352-18831d65a3da None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:28.557103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c73ff623-6373-4f7d-b352-18831d65a3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:28.652460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:28.807926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69b6ec-a483-4c26-903a-9435dbe3e91b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:28.809494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3f69b6ec-a483-4c26-903a-9435dbe3e91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:28.958049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69b6ec-a483-4c26-903a-9435dbe3e91b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:28.970183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:29.074091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf33bf-0cf3-4b69-8a5a-6976f4ec8ffc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:29.074695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf33bf-0cf3-4b69-8a5a-6976f4ec8ffc service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:29.074968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf33bf-0cf3-4b69-8a5a-6976f4ec8ffc service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:29.075239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf33bf-0cf3-4b69-8a5a-6976f4ec8ffc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:29.076874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3fcf33bf-0cf3-4b69-8a5a-6976f4ec8ffc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:29.164898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf33bf-0cf3-4b69-8a5a-6976f4ec8ffc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:29.166490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf33bf-0cf3-4b69-8a5a-6976f4ec8ffc service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:29.184904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 721/1422] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:30.049408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893e0d7-69ff-4b30-9574-b6f8fa30e8f0 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:30.049924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893e0d7-69ff-4b30-9574-b6f8fa30e8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:30.049924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893e0d7-69ff-4b30-9574-b6f8fa30e8f0 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:30.051936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:30.057192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff451f9-bd86-4a6f-8090-547d3650ef92 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:30.057643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff451f9-bd86-4a6f-8090-547d3650ef92 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:30.057748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff451f9-bd86-4a6f-8090-547d3650ef92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:30.077542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:30.133210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8ff451f9-bd86-4a6f-8090-547d3650ef92 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:30.134402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8ff451f9-bd86-4a6f-8090-547d3650ef92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:30.228480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:30.377304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893e0d7-69ff-4b30-9574-b6f8fa30e8f0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:30.378915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9893e0d7-69ff-4b30-9574-b6f8fa30e8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:30.488929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:30.523655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893e0d7-69ff-4b30-9574-b6f8fa30e8f0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:30.636412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad1689-9592-4a74-ab4d-702abe8bb573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:30.636977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad1689-9592-4a74-ab4d-702abe8bb573 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:30.637252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad1689-9592-4a74-ab4d-702abe8bb573 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:30.637519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad1689-9592-4a74-ab4d-702abe8bb573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:30.639135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8dad1689-9592-4a74-ab4d-702abe8bb573 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:30.714657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:30.728804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad1689-9592-4a74-ab4d-702abe8bb573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:30.730386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad1689-9592-4a74-ab4d-702abe8bb573 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:30.748418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 703/1425] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:31.610532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eaecb-6e21-4de8-b517-1542693abe8c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:31.611097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eaecb-6e21-4de8-b517-1542693abe8c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:31.611097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6eaecb-6e21-4de8-b517-1542693abe8c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:31.612465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:31.618008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb61cd-3a72-4c6d-8ecb-4ce73a74b1c3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:31.618464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb61cd-3a72-4c6d-8ecb-4ce73a74b1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:31.618572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb61cd-3a72-4c6d-8ecb-4ce73a74b1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:31.703576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6cfb61cd-3a72-4c6d-8ecb-4ce73a74b1c3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:31.704801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6cfb61cd-3a72-4c6d-8ecb-4ce73a74b1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:31.804139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:31.956278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6eaecb-6e21-4de8-b517-1542693abe8c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:31.957848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ae6eaecb-6e21-4de8-b517-1542693abe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:32.049015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:32.113789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6eaecb-6e21-4de8-b517-1542693abe8c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:32.225550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb098af7-9e17-4edb-b76a-c531d8e702b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:32.226068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb098af7-9e17-4edb-b76a-c531d8e702b6 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:32.226340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb098af7-9e17-4edb-b76a-c531d8e702b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:32.226605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb098af7-9e17-4edb-b76a-c531d8e702b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:32.228201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cb098af7-9e17-4edb-b76a-c531d8e702b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:32.275421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:32.318079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb098af7-9e17-4edb-b76a-c531d8e702b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:32.319697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb098af7-9e17-4edb-b76a-c531d8e702b6 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:32.338108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 724/1428] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:33.151890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c98b4b-bfc1-4aa3-b7aa-2fc97e0e0920 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:33.152594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c98b4b-bfc1-4aa3-b7aa-2fc97e0e0920 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:33.152594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c98b4b-bfc1-4aa3-b7aa-2fc97e0e0920 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:33.153853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:33.158701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d725d8-01b0-4be5-bfff-49673911c5c8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:33.159140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d725d8-01b0-4be5-bfff-49673911c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:33.159242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d725d8-01b0-4be5-bfff-49673911c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:33.236362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-37d725d8-01b0-4be5-bfff-49673911c5c8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:33.238115 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-37d725d8-01b0-4be5-bfff-49673911c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:33.336595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:33.488535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c98b4b-bfc1-4aa3-b7aa-2fc97e0e0920 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:33.490036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-08c98b4b-bfc1-4aa3-b7aa-2fc97e0e0920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:33.538461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:33.634666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c98b4b-bfc1-4aa3-b7aa-2fc97e0e0920 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:33.762629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c72e7-d192-46df-b636-fe4c9b1b4eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:33.763149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c72e7-d192-46df-b636-fe4c9b1b4eb4 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:33.763428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c72e7-d192-46df-b636-fe4c9b1b4eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:33.763694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c72e7-d192-46df-b636-fe4c9b1b4eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:33.765323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fe0c72e7-d192-46df-b636-fe4c9b1b4eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:33.772339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:33.857850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c72e7-d192-46df-b636-fe4c9b1b4eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:33.859453 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c72e7-d192-46df-b636-fe4c9b1b4eb4 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:33.877777 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 706/1431] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:34.712845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978da88-bfa0-4495-8b1f-524271b7f5cf None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:34.713376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978da88-bfa0-4495-8b1f-524271b7f5cf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:34.713482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978da88-bfa0-4495-8b1f-524271b7f5cf None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:34.715265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:34.720245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150c55e-8c43-4c7d-aa71-81294fa4048f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:34.720689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150c55e-8c43-4c7d-aa71-81294fa4048f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:34.720835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150c55e-8c43-4c7d-aa71-81294fa4048f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:34.799744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1150c55e-8c43-4c7d-aa71-81294fa4048f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:34.801525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1150c55e-8c43-4c7d-aa71-81294fa4048f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:34.869124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:34.898010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:35.056805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978da88-bfa0-4495-8b1f-524271b7f5cf None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:35.058386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7978da88-bfa0-4495-8b1f-524271b7f5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:35.076077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:35.202353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978da88-bfa0-4495-8b1f-524271b7f5cf None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:35.289311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:35.315269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bffa5-4165-425e-b00b-7aa1e99812e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:35.315937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bffa5-4165-425e-b00b-7aa1e99812e0 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:35.316456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bffa5-4165-425e-b00b-7aa1e99812e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:35.316826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bffa5-4165-425e-b00b-7aa1e99812e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:35.318437 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-052bffa5-4165-425e-b00b-7aa1e99812e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:35.408371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bffa5-4165-425e-b00b-7aa1e99812e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:35.410790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bffa5-4165-425e-b00b-7aa1e99812e0 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:35.428151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 727/1434] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:36.241047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13c40-bf11-488b-b7df-c879d51e5501 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:36.241627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13c40-bf11-488b-b7df-c879d51e5501 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:36.241627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13c40-bf11-488b-b7df-c879d51e5501 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:36.242943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:36.247750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ab5e-2553-4692-964c-1c2f00f33565 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:36.248146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ab5e-2553-4692-964c-1c2f00f33565 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:36.248297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ab5e-2553-4692-964c-1c2f00f33565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:36.323261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-34c1ab5e-2553-4692-964c-1c2f00f33565 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:36.324537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-34c1ab5e-2553-4692-964c-1c2f00f33565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:36.370817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:36.440533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:36.590165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f13c40-bf11-488b-b7df-c879d51e5501 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:36.591698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e0f13c40-bf11-488b-b7df-c879d51e5501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:36.741069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f13c40-bf11-488b-b7df-c879d51e5501 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:36.794053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:36.855093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782bf85-5984-4b6d-b0c8-b88346a742d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:36.855704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bf85-5984-4b6d-b0c8-b88346a742d5 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:36.855982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bf85-5984-4b6d-b0c8-b88346a742d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:36.856255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782bf85-5984-4b6d-b0c8-b88346a742d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:36.857896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e782bf85-5984-4b6d-b0c8-b88346a742d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:36.948389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782bf85-5984-4b6d-b0c8-b88346a742d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:36.950007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782bf85-5984-4b6d-b0c8-b88346a742d5 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:36.967841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 709/1437] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:37.783271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba1a63-9d68-4e14-8987-3449f233f796 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:37.783852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba1a63-9d68-4e14-8987-3449f233f796 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:37.783852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba1a63-9d68-4e14-8987-3449f233f796 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:37.786378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:07:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:37.795076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67bb6a-321d-4523-8157-d779c78c656e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:37.795521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67bb6a-321d-4523-8157-d779c78c656e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:37.795642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67bb6a-321d-4523-8157-d779c78c656e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:37.871961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7c67bb6a-321d-4523-8157-d779c78c656e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:37.873231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7c67bb6a-321d-4523-8157-d779c78c656e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:37.875353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:37.969396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:38.121089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba1a63-9d68-4e14-8987-3449f233f796 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:38.122522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-65ba1a63-9d68-4e14-8987-3449f233f796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:38.272259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba1a63-9d68-4e14-8987-3449f233f796 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:38.299208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:38.384487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939df6b-6d84-4f04-a3e7-74753b5763d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:38.385010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939df6b-6d84-4f04-a3e7-74753b5763d2 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:38.385284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939df6b-6d84-4f04-a3e7-74753b5763d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:38.385553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939df6b-6d84-4f04-a3e7-74753b5763d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:38.387189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5939df6b-6d84-4f04-a3e7-74753b5763d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:38.596128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939df6b-6d84-4f04-a3e7-74753b5763d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:38.598557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939df6b-6d84-4f04-a3e7-74753b5763d2 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:38.617406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 730/1440] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:40.704404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c68f99-f649-407c-b5db-8d7349b0916a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:40.704999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c68f99-f649-407c-b5db-8d7349b0916a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:40.704999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c68f99-f649-407c-b5db-8d7349b0916a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:40.706835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:40.711969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7ba71-5bb0-4482-bcb1-a3c9c4908408 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:40.712407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7ba71-5bb0-4482-bcb1-a3c9c4908408 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:40.712542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7ba71-5bb0-4482-bcb1-a3c9c4908408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:40.752642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:40.777999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-55b7ba71-5bb0-4482-bcb1-a3c9c4908408 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:40.779198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-55b7ba71-5bb0-4482-bcb1-a3c9c4908408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:40.877026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:40.915998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c68f99-f649-407c-b5db-8d7349b0916a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:40.917585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-15c68f99-f649-407c-b5db-8d7349b0916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:41.025121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c68f99-f649-407c-b5db-8d7349b0916a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:41.060670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:41.105312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4192d-dccf-476f-b021-313a709fa570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:41.105951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4192d-dccf-476f-b021-313a709fa570 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:41.106230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4192d-dccf-476f-b021-313a709fa570 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:41.106501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4192d-dccf-476f-b021-313a709fa570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:41.108110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-49e4192d-dccf-476f-b021-313a709fa570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:41.225207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4192d-dccf-476f-b021-313a709fa570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:41.226640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4192d-dccf-476f-b021-313a709fa570 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:41.246153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 712/1443] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:43.423745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e14b4-9920-4693-a690-a446f063f3bd None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:43.424327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e14b4-9920-4693-a690-a446f063f3bd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:43.424327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e14b4-9920-4693-a690-a446f063f3bd None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:43.426210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:43.431761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3fcbf-35d8-495e-bbf7-a6334dcbe8d6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:43.432247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3fcbf-35d8-495e-bbf7-a6334dcbe8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:43.432363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3fcbf-35d8-495e-bbf7-a6334dcbe8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:43.617272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-adf3fcbf-35d8-495e-bbf7-a6334dcbe8d6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:43.618391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-adf3fcbf-35d8-495e-bbf7-a6334dcbe8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:43.631500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:43.714421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:43.753694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e14b4-9920-4693-a690-a446f063f3bd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:43.755330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-267e14b4-9920-4693-a690-a446f063f3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:43.858944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e14b4-9920-4693-a690-a446f063f3bd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:43.872171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:43.934047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07060c73-b7f9-4576-a730-50113aa3958b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:43.934641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060c73-b7f9-4576-a730-50113aa3958b service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:43.934916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060c73-b7f9-4576-a730-50113aa3958b service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:43.935189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07060c73-b7f9-4576-a730-50113aa3958b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:43.936814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07060c73-b7f9-4576-a730-50113aa3958b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:44.043125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07060c73-b7f9-4576-a730-50113aa3958b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:44.044571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07060c73-b7f9-4576-a730-50113aa3958b service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:44.062511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 733/1446] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:46.483733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b858e61-3a45-4ad7-a5ff-98f10c17e197 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:46.484282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b858e61-3a45-4ad7-a5ff-98f10c17e197 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:46.484282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b858e61-3a45-4ad7-a5ff-98f10c17e197 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:46.485648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:46.491722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c826b-bc06-4c4d-937f-5f76b01741e8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:46.492118 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c826b-bc06-4c4d-937f-5f76b01741e8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:46.492273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c826b-bc06-4c4d-937f-5f76b01741e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:46.539264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:46.558207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-618c826b-bc06-4c4d-937f-5f76b01741e8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:46.559416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-618c826b-bc06-4c4d-937f-5f76b01741e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:46.654038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:46.695282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b858e61-3a45-4ad7-a5ff-98f10c17e197 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:46.696894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3b858e61-3a45-4ad7-a5ff-98f10c17e197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:46.797664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b858e61-3a45-4ad7-a5ff-98f10c17e197 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:46.843193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:46.878849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe4281-a433-49af-9ad7-7b1a9911d424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:46.879469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe4281-a433-49af-9ad7-7b1a9911d424 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:46.879747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe4281-a433-49af-9ad7-7b1a9911d424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:46.880015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe4281-a433-49af-9ad7-7b1a9911d424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:46.881698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ecbe4281-a433-49af-9ad7-7b1a9911d424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:46.991836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe4281-a433-49af-9ad7-7b1a9911d424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:46.993225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe4281-a433-49af-9ad7-7b1a9911d424 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:47.010968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 715/1449] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:49.218075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454cd3f-9e84-4b08-a89d-a0341ab7824b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:49.218635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454cd3f-9e84-4b08-a89d-a0341ab7824b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:49.218635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454cd3f-9e84-4b08-a89d-a0341ab7824b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:49.219957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:49.226161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d75b4-6d1e-4eee-82e4-b41b5bdccb75 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:49.226611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d75b4-6d1e-4eee-82e4-b41b5bdccb75 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:49.226719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d75b4-6d1e-4eee-82e4-b41b5bdccb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:49.293673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-969d75b4-6d1e-4eee-82e4-b41b5bdccb75 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:49.294880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-969d75b4-6d1e-4eee-82e4-b41b5bdccb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:49.316398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:49.393829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:49.435031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454cd3f-9e84-4b08-a89d-a0341ab7824b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:49.437893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a454cd3f-9e84-4b08-a89d-a0341ab7824b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:49.543787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454cd3f-9e84-4b08-a89d-a0341ab7824b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:49.563925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:49.617447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdc2b0-1bd8-4141-840f-96c6f6d3972a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:49.618044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdc2b0-1bd8-4141-840f-96c6f6d3972a service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:49.618326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdc2b0-1bd8-4141-840f-96c6f6d3972a service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:49.618599 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdc2b0-1bd8-4141-840f-96c6f6d3972a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:49.620200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f5bdc2b0-1bd8-4141-840f-96c6f6d3972a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:49.728737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdc2b0-1bd8-4141-840f-96c6f6d3972a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:49.730179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdc2b0-1bd8-4141-840f-96c6f6d3972a service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:49.748659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 736/1452] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:51.841472 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cd015-aa42-4fb0-840f-80be28be0376 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:51.842028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cd015-aa42-4fb0-840f-80be28be0376 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:51.842028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cd015-aa42-4fb0-840f-80be28be0376 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:51.843460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:51.849214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed9ef9-7ee3-42f7-87f6-6ef4ad855145 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:51.849658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed9ef9-7ee3-42f7-87f6-6ef4ad855145 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:51.849784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed9ef9-7ee3-42f7-87f6-6ef4ad855145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:51.919471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:51.922195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7aed9ef9-7ee3-42f7-87f6-6ef4ad855145 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:51.924024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7aed9ef9-7ee3-42f7-87f6-6ef4ad855145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:52.022702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:52.062616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cd015-aa42-4fb0-840f-80be28be0376 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:52.064094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ed2cd015-aa42-4fb0-840f-80be28be0376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:52.172158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cd015-aa42-4fb0-840f-80be28be0376 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:52.222503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:52.247970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa45363-c4a0-472d-8794-a5b1f6c7fab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:52.248647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa45363-c4a0-472d-8794-a5b1f6c7fab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:52.248932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa45363-c4a0-472d-8794-a5b1f6c7fab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:52.249208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa45363-c4a0-472d-8794-a5b1f6c7fab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:52.250819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6fa45363-c4a0-472d-8794-a5b1f6c7fab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:52.361306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa45363-c4a0-472d-8794-a5b1f6c7fab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:52.363016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa45363-c4a0-472d-8794-a5b1f6c7fab0 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:52.381289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 718/1455] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:54.434663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a3d40-bba8-4f72-a21c-eea220db3837 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:54.435191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a3d40-bba8-4f72-a21c-eea220db3837 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:54.435191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a3d40-bba8-4f72-a21c-eea220db3837 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:54.436566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:54.442188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bf5ab-7c2f-4873-ab8c-19a09b706f45 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:54.442591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bf5ab-7c2f-4873-ab8c-19a09b706f45 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:54.442732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bf5ab-7c2f-4873-ab8c-19a09b706f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:54.509195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c34bf5ab-7c2f-4873-ab8c-19a09b706f45 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:54.510390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c34bf5ab-7c2f-4873-ab8c-19a09b706f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:54.539013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:54.606202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:54.644861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a3d40-bba8-4f72-a21c-eea220db3837 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:54.646409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8b8a3d40-bba8-4f72-a21c-eea220db3837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:54.753019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a3d40-bba8-4f72-a21c-eea220db3837 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:54.781086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:54.827711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647dc42-9f83-4a33-abd7-9c1bf63d2ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:54.828316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647dc42-9f83-4a33-abd7-9c1bf63d2ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:54.828627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647dc42-9f83-4a33-abd7-9c1bf63d2ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:54.828897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647dc42-9f83-4a33-abd7-9c1bf63d2ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:54.830507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c647dc42-9f83-4a33-abd7-9c1bf63d2ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:54.938351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647dc42-9f83-4a33-abd7-9c1bf63d2ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:54.939799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647dc42-9f83-4a33-abd7-9c1bf63d2ee3 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:54.959200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 739/1458] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:57.059799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eecba-86bb-4b40-be7d-461acba9c8a5 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:57.060336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eecba-86bb-4b40-be7d-461acba9c8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:57.060336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879eecba-86bb-4b40-be7d-461acba9c8a5 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:57.061703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:57.067300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dc54f-1489-4cc5-9e73-17261858dfa6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:57.067828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dc54f-1489-4cc5-9e73-17261858dfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:57.067828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dc54f-1489-4cc5-9e73-17261858dfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:57.136463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-791dc54f-1489-4cc5-9e73-17261858dfa6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:57.137707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-791dc54f-1489-4cc5-9e73-17261858dfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:57.146565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:57.235123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:07:57.273888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879eecba-86bb-4b40-be7d-461acba9c8a5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:57.275428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-879eecba-86bb-4b40-be7d-461acba9c8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:57.380037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879eecba-86bb-4b40-be7d-461acba9c8a5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:57.436117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:57.454488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af5404-9dbe-440e-b10b-d7513d8dda8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:07:57.455090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5404-9dbe-440e-b10b-d7513d8dda8f service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:57.455372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5404-9dbe-440e-b10b-d7513d8dda8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:57.455660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5404-9dbe-440e-b10b-d7513d8dda8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:57.457630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-73af5404-9dbe-440e-b10b-d7513d8dda8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:57.565917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af5404-9dbe-440e-b10b-d7513d8dda8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:07:57.567347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af5404-9dbe-440e-b10b-d7513d8dda8f service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:07:57.585026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 721/1461] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:07:59.636560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e3668-0d24-4bbc-a48b-04ed3f5ef35a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:59.637088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e3668-0d24-4bbc-a48b-04ed3f5ef35a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:59.637088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e3668-0d24-4bbc-a48b-04ed3f5ef35a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:59.638518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:07:59.644256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9860ebf-e2b1-4ae1-8252-5d2ef077bcf8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:07:59.644716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9860ebf-e2b1-4ae1-8252-5d2ef077bcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:07:59.644845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9860ebf-e2b1-4ae1-8252-5d2ef077bcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:07:59.710935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a9860ebf-e2b1-4ae1-8252-5d2ef077bcf8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:07:59.712135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a9860ebf-e2b1-4ae1-8252-5d2ef077bcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:59.750527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:07:59.809667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:07:59.848926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e3668-0d24-4bbc-a48b-04ed3f5ef35a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:07:59.850617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-663e3668-0d24-4bbc-a48b-04ed3f5ef35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:07:59.961337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e3668-0d24-4bbc-a48b-04ed3f5ef35a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:07:59.998186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:00.033482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7f828-0bfd-4b27-9a6f-aa5620af1cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:00.034130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f828-0bfd-4b27-9a6f-aa5620af1cdd service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:00.034378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f828-0bfd-4b27-9a6f-aa5620af1cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:00.034644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f828-0bfd-4b27-9a6f-aa5620af1cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:00.036247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-13e7f828-0bfd-4b27-9a6f-aa5620af1cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:00.144317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7f828-0bfd-4b27-9a6f-aa5620af1cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:00.145774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7f828-0bfd-4b27-9a6f-aa5620af1cdd service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:00.163276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 742/1464] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:07:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:02.186312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3c440-1f8f-49b4-986c-b15e61245990 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:02.186856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3c440-1f8f-49b4-986c-b15e61245990 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:02.186856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3c440-1f8f-49b4-986c-b15e61245990 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:02.188200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:02.193968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4957e45-ae10-4c28-8426-5512577b8cf1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:02.194356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4957e45-ae10-4c28-8426-5512577b8cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:02.194517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4957e45-ae10-4c28-8426-5512577b8cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:02.268403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b4957e45-ae10-4c28-8426-5512577b8cf1 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:02.269843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b4957e45-ae10-4c28-8426-5512577b8cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:02.287615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:02.364672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:02.402890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3c440-1f8f-49b4-986c-b15e61245990 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:02.404499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-42f3c440-1f8f-49b4-986c-b15e61245990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:02.512006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3c440-1f8f-49b4-986c-b15e61245990 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:02.596878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:02.603968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c57ab6-3fee-406e-a04c-4392476ae545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:02.604600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57ab6-3fee-406e-a04c-4392476ae545 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:02.604875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57ab6-3fee-406e-a04c-4392476ae545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:02.605138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57ab6-3fee-406e-a04c-4392476ae545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:02.606712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-39c57ab6-3fee-406e-a04c-4392476ae545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:02.714543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c57ab6-3fee-406e-a04c-4392476ae545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:02.715969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c57ab6-3fee-406e-a04c-4392476ae545 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:02.733825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 724/1467] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:04.769746 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292e11-56c7-47c4-94fb-54fe870aee42 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:04.770355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292e11-56c7-47c4-94fb-54fe870aee42 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:04.770355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292e11-56c7-47c4-94fb-54fe870aee42 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:04.771629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:04.777022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec2fa5-1606-4ad3-b162-08dc61f0af6b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:04.777462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec2fa5-1606-4ad3-b162-08dc61f0af6b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:04.777566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec2fa5-1606-4ad3-b162-08dc61f0af6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:04.844722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4aec2fa5-1606-4ad3-b162-08dc61f0af6b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:04.845931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4aec2fa5-1606-4ad3-b162-08dc61f0af6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:04.897031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:04.948545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:04.986655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40292e11-56c7-47c4-94fb-54fe870aee42 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:04.988758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-40292e11-56c7-47c4-94fb-54fe870aee42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:05.091142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40292e11-56c7-47c4-94fb-54fe870aee42 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:05.137749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:05.163951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dcac3-1a81-417d-aa3f-fd29701151ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:05.164584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dcac3-1a81-417d-aa3f-fd29701151ab service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:05.164858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dcac3-1a81-417d-aa3f-fd29701151ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:05.165120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dcac3-1a81-417d-aa3f-fd29701151ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:05.166694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d13dcac3-1a81-417d-aa3f-fd29701151ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:05.286546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13dcac3-1a81-417d-aa3f-fd29701151ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:05.287909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13dcac3-1a81-417d-aa3f-fd29701151ab service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:05.305836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 745/1470] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:14.946714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=22 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:15.303441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=66 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:25.094969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9497e34-afe5-48a3-b46c-dd46f6b4936f None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:25.095499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9497e34-afe5-48a3-b46c-dd46f6b4936f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:25.095499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9497e34-afe5-48a3-b46c-dd46f6b4936f None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:25.096892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 726/1471] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:25.102172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1348f-edae-4a09-85fa-651178faae41 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:25.102615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1348f-edae-4a09-85fa-651178faae41 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:25.102719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1348f-edae-4a09-85fa-651178faae41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:25.169178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a1b1348f-edae-4a09-85fa-651178faae41 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:25.170376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a1b1348f-edae-4a09-85fa-651178faae41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:25.267193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 746/1472] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:25.353049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0a2-0f61-4187-8d2d-8594c78fde10 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:25.353324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0a2-0f61-4187-8d2d-8594c78fde10 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:25.353595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0a2-0f61-4187-8d2d-8594c78fde10 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:25.354902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 727/1473] 10.222.0.70 () {54 vars in 967 bytes} [Tue Aug 9 19:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:25.359787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf66ce-edab-4117-be4a-5c8674af149e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:25.360078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf66ce-edab-4117-be4a-5c8674af149e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:25.360353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf66ce-edab-4117-be4a-5c8674af149e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:25.440732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f0cf66ce-edab-4117-be4a-5c8674af149e None None] MFA Rules not processed for user `92a88817f3a4402290e9358826a63904`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:25.441944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f0cf66ce-edab-4117-be4a-5c8674af149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:25.462155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:25.516711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 747/1474] 10.222.0.70 () {58 vars in 1064 bytes} [Tue Aug 9 19:08:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:08:25.524344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ed0a2-0f61-4187-8d2d-8594c78fde10 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:25.525908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-975ed0a2-0f61-4187-8d2d-8594c78fde10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:25.628203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ed0a2-0f61-4187-8d2d-8594c78fde10 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:25.699863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f0f95-7601-4572-b788-93952c1840d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a88817f3a4402290e9358826a63904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:25.700477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0f95-7601-4572-b788-93952c1840d2 service glance] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:25.700758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0f95-7601-4572-b788-93952c1840d2 service glance] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:25.701020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f0f95-7601-4572-b788-93952c1840d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:25.702602 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-230f0f95-7601-4572-b788-93952c1840d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:25.755374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:25.809375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f0f95-7601-4572-b788-93952c1840d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:25.810795 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f0f95-7601-4572-b788-93952c1840d2 service glance] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:25.830191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 728/1475] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:27.967698 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad004cf3-e010-4bc6-aba0-b82a4ca401f0 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:27.968287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad004cf3-e010-4bc6-aba0-b82a4ca401f0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:27.968287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad004cf3-e010-4bc6-aba0-b82a4ca401f0 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:27.969634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 748/1476] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:27.975771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c705b-8fa1-4b09-a041-14d37a2847ff None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:27.976201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c705b-8fa1-4b09-a041-14d37a2847ff None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:27.976308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c705b-8fa1-4b09-a041-14d37a2847ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:28.041133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5c4c705b-8fa1-4b09-a041-14d37a2847ff None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:28.042321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5c4c705b-8fa1-4b09-a041-14d37a2847ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:28.140115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 729/1477] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:28.164183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad004cf3-e010-4bc6-aba0-b82a4ca401f0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:28.165755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ad004cf3-e010-4bc6-aba0-b82a4ca401f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:28.289263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad004cf3-e010-4bc6-aba0-b82a4ca401f0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:28.337461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:28.381780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59256df-74fd-448e-b44d-0918f049b8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:28.382381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59256df-74fd-448e-b44d-0918f049b8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:28.382658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59256df-74fd-448e-b44d-0918f049b8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:28.382930 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59256df-74fd-448e-b44d-0918f049b8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:28.383603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59256df-74fd-448e-b44d-0918f049b8d3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:28.385416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-c59256df-74fd-448e-b44d-0918f049b8d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:08:28.385416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Aug 09 19:08:28.385416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: Aug 09 19:08:28.386558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59256df-74fd-448e-b44d-0918f049b8d3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:28.407777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 749/1478] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Aug 9 19:08:28 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:08:30.453181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9f8d-19ff-492b-92c2-b57a496fd1e3 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:30.453724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9f8d-19ff-492b-92c2-b57a496fd1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:30.453724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9f8d-19ff-492b-92c2-b57a496fd1e3 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:30.455980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 730/1479] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:30.461246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0962-d726-49f1-8cb9-f0dd45b8ba69 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:30.461676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0962-d726-49f1-8cb9-f0dd45b8ba69 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:30.461802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0962-d726-49f1-8cb9-f0dd45b8ba69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:30.526872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-24ea0962-d726-49f1-8cb9-f0dd45b8ba69 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:30.528078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-24ea0962-d726-49f1-8cb9-f0dd45b8ba69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:30.621479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 750/1480] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:30.665118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c9f8d-19ff-492b-92c2-b57a496fd1e3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:30.666661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cf8c9f8d-19ff-492b-92c2-b57a496fd1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:30.676735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:30.811099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c9f8d-19ff-492b-92c2-b57a496fd1e3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:30.891183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:30.932713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b8d37-3df3-499d-9e83-ba5ea0ccb99c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:30.933240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b8d37-3df3-499d-9e83-ba5ea0ccb99c service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:30.933517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b8d37-3df3-499d-9e83-ba5ea0ccb99c service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:30.933787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b8d37-3df3-499d-9e83-ba5ea0ccb99c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:30.935404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dc1b8d37-3df3-499d-9e83-ba5ea0ccb99c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:31.023931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b8d37-3df3-499d-9e83-ba5ea0ccb99c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:31.025539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b8d37-3df3-499d-9e83-ba5ea0ccb99c service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:31.042969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 731/1481] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:33.227569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937410-962e-4a7d-9ee7-b87a7882599f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:33.228135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937410-962e-4a7d-9ee7-b87a7882599f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:33.228135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937410-962e-4a7d-9ee7-b87a7882599f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:33.229481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 751/1482] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:33.234695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb513d3-d7ad-4944-9e08-bf3d9d1f9720 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:33.235147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb513d3-d7ad-4944-9e08-bf3d9d1f9720 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:33.235244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb513d3-d7ad-4944-9e08-bf3d9d1f9720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:33.303436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-edb513d3-d7ad-4944-9e08-bf3d9d1f9720 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:33.304915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-edb513d3-d7ad-4944-9e08-bf3d9d1f9720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:33.352659 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:33.403461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 732/1483] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:33.441190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8937410-962e-4a7d-9ee7-b87a7882599f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:33.442712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e8937410-962e-4a7d-9ee7-b87a7882599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:33.467067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:33.555688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8937410-962e-4a7d-9ee7-b87a7882599f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:33.627553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e9073-86f0-4cac-9339-c83ed487fa88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:33.628149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e9073-86f0-4cac-9339-c83ed487fa88 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:33.628423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e9073-86f0-4cac-9339-c83ed487fa88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:33.628719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e9073-86f0-4cac-9339-c83ed487fa88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:33.630316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8f2e9073-86f0-4cac-9339-c83ed487fa88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:33.703724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:33.739719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e9073-86f0-4cac-9339-c83ed487fa88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:33.741223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e9073-86f0-4cac-9339-c83ed487fa88 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:33.759173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 752/1484] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:35.860691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82976e39-730e-4bb0-8ddd-bf01b3f216da None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:35.861278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82976e39-730e-4bb0-8ddd-bf01b3f216da None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:35.862122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82976e39-730e-4bb0-8ddd-bf01b3f216da None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:35.863471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 733/1485] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:35.868875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a04a5-b302-4240-9059-b756ef75a89a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:35.869302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a04a5-b302-4240-9059-b756ef75a89a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:35.869427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a04a5-b302-4240-9059-b756ef75a89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:35.935868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8c1a04a5-b302-4240-9059-b756ef75a89a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:35.937095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8c1a04a5-b302-4240-9059-b756ef75a89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:36.035873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 753/1486] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:36.073979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82976e39-730e-4bb0-8ddd-bf01b3f216da None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:36.075505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-82976e39-730e-4bb0-8ddd-bf01b3f216da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:36.182173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82976e39-730e-4bb0-8ddd-bf01b3f216da None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:36.257865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:36.259561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec158-1f35-4917-b63d-a42e5aade3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:36.260169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec158-1f35-4917-b63d-a42e5aade3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:36.260657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec158-1f35-4917-b63d-a42e5aade3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:36.261146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec158-1f35-4917-b63d-a42e5aade3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:36.262961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1f3ec158-1f35-4917-b63d-a42e5aade3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:36.374356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ec158-1f35-4917-b63d-a42e5aade3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:36.375833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ec158-1f35-4917-b63d-a42e5aade3e2 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:36.395306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 734/1487] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:38.614161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d298985-e9ca-4d28-9c2d-cc03957ba3ab None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:38.614694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d298985-e9ca-4d28-9c2d-cc03957ba3ab None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:38.614694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d298985-e9ca-4d28-9c2d-cc03957ba3ab None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:38.616026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 754/1488] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:38.621259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7402ee-350e-44f2-ac1f-48bf069c9f66 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:38.621709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7402ee-350e-44f2-ac1f-48bf069c9f66 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:38.621822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7402ee-350e-44f2-ac1f-48bf069c9f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:38.688315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9b7402ee-350e-44f2-ac1f-48bf069c9f66 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:38.690346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9b7402ee-350e-44f2-ac1f-48bf069c9f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:38.743036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:38.788542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 735/1489] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:38.828950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d298985-e9ca-4d28-9c2d-cc03957ba3ab None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:38.831259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5d298985-e9ca-4d28-9c2d-cc03957ba3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:38.866572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:38.936171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d298985-e9ca-4d28-9c2d-cc03957ba3ab None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:39.009976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3399-dabc-4924-a563-472c10beaa66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:39.010643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3399-dabc-4924-a563-472c10beaa66 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:39.010917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3399-dabc-4924-a563-472c10beaa66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:39.011183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3399-dabc-4924-a563-472c10beaa66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:39.012813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5b8a3399-dabc-4924-a563-472c10beaa66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:39.101675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:39.131590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3399-dabc-4924-a563-472c10beaa66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:39.132996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3399-dabc-4924-a563-472c10beaa66 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:39.151399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 755/1490] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:41.588141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626728b6-058e-409c-9264-10a36ae4e2a6 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:41.588717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626728b6-058e-409c-9264-10a36ae4e2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:41.588717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626728b6-058e-409c-9264-10a36ae4e2a6 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:41.590059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 736/1491] 10.222.0.70 () {54 vars in 927 bytes} [Tue Aug 9 19:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:08:41.595414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb3faf-a992-42e1-a80e-968aa2616179 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:41.595841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb3faf-a992-42e1-a80e-968aa2616179 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:41.596041 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb3faf-a992-42e1-a80e-968aa2616179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:41.660369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-54cb3faf-a992-42e1-a80e-968aa2616179 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:08:41.661598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-54cb3faf-a992-42e1-a80e-968aa2616179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:41.756881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 756/1492] 10.222.0.70 () {58 vars in 1024 bytes} [Tue Aug 9 19:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:08:41.796240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626728b6-058e-409c-9264-10a36ae4e2a6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:08:41.797853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-626728b6-058e-409c-9264-10a36ae4e2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:41.900762 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626728b6-058e-409c-9264-10a36ae4e2a6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:08:41.974359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92911be-ed71-4b98-8f05-f5f53d0c4fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:08:41.974891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92911be-ed71-4b98-8f05-f5f53d0c4fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:08:41.975176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92911be-ed71-4b98-8f05-f5f53d0c4fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:08:41.975444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92911be-ed71-4b98-8f05-f5f53d0c4fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:08:41.977087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b92911be-ed71-4b98-8f05-f5f53d0c4fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:08:41.986521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:42.089134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92911be-ed71-4b98-8f05-f5f53d0c4fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:08:42.090563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92911be-ed71-4b98-8f05-f5f53d0c4fe5 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:08:42.108531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 737/1493] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:08:51.754987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=82 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:08:52.106337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=50 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:12.951621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac2e0f-6811-4dcc-9ffb-0f6e7320cc8e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:12.952236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac2e0f-6811-4dcc-9ffb-0f6e7320cc8e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:12.952236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac2e0f-6811-4dcc-9ffb-0f6e7320cc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:13.026786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b7ac2e0f-6811-4dcc-9ffb-0f6e7320cc8e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:10:13.027996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b7ac2e0f-6811-4dcc-9ffb-0f6e7320cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:13.126281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 757/1494] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:10:13.134775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dff87-6de3-4541-8314-c0e7f3fca7dc None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:13.135266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dff87-6de3-4541-8314-c0e7f3fca7dc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:13.135374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dff87-6de3-4541-8314-c0e7f3fca7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:13.211599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d90dff87-6de3-4541-8314-c0e7f3fca7dc None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:10:13.212758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d90dff87-6de3-4541-8314-c0e7f3fca7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:13.307752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 738/1495] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:10:13.313023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac2e0f-6811-4dcc-9ffb-0f6e7320cc8e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:13.314360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b7ac2e0f-6811-4dcc-9ffb-0f6e7320cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:13.325791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac2e0f-6811-4dcc-9ffb-0f6e7320cc8e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:13.386383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:13.421028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:13.421712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:13.422399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:13.422642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:13.423285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:10:13.423578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:13.424610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:10:13.424610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Aug 09 19:10:13.424610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: Aug 09 19:10:13.425486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:13.436732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 758/1496] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:10:13.447001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dff87-6de3-4541-8314-c0e7f3fca7dc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:13.456169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dff87-6de3-4541-8314-c0e7f3fca7dc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:13.516753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:13.551674 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:13.552279 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:13.552586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:13.552855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:13.553736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:13.555439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:10:13.555439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Aug 09 19:10:13.555439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: Aug 09 19:10:13.556642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:13.611000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 739/1497] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:10:13 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:10:13.615211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:13.616744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:13.649248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056e5f7-0740-4aab-b678-4dc06cc29d5f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:13.741939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:13.742543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:13.742818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:13.743088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:13.743703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:13.744744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:10:13.744744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Aug 09 19:10:13.744744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: Aug 09 19:10:13.745942 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:13.791167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 759/1498] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:10:13 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:10:13.795516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:13.797675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:13.860740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed885d-e1e0-46d7-8928-3a2c078436d3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:13.947570 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:13.959044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:13.959716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:13.959999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:13.960273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:13.960953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:13.961997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:10:13.961997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Aug 09 19:10:13.961997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: Aug 09 19:10:13.962877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:13.975862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 740/1499] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:10:13.979996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:13.981583 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:13.994067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295af20-5657-441c-9278-8e2c7c235d75 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:14.018537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:14.089923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f4a15-fbbb-4a86-97c5-e6f6cab755a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:14.090437 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4a15-fbbb-4a86-97c5-e6f6cab755a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:14.090711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4a15-fbbb-4a86-97c5-e6f6cab755a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:14.090979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f4a15-fbbb-4a86-97c5-e6f6cab755a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:14.091614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f4a15-fbbb-4a86-97c5-e6f6cab755a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:14.092677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-168f4a15-fbbb-4a86-97c5-e6f6cab755a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:10:14.092677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Aug 09 19:10:14.092677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: Aug 09 19:10:14.093556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f4a15-fbbb-4a86-97c5-e6f6cab755a4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:14.106112 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 760/1500] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:10:14.111422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:14.113039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:14.147096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f4a32-6133-4b61-b12e-c9d2ca2cf208 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:14.241307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcc305-f523-4611-874d-db5fab39e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:14.241965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcc305-f523-4611-874d-db5fab39e1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:14.242243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcc305-f523-4611-874d-db5fab39e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:14.242519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcc305-f523-4611-874d-db5fab39e1e5 admin admin] PATH_INFO: `/v3/projects/e43c25434e5f4929b87abde75ea86e8f/users/f8e6315af02143cb91ec6c80129ed96e/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:14.263944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcc305-f523-4611-874d-db5fab39e1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43c25434e5f4929b87abde75ea86e8f, user_id=f8e6315af02143cb91ec6c80129ed96e, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:14.264998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-4adcc305-f523-4611-874d-db5fab39e1e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:10:14.264998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Aug 09 19:10:14.264998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: Aug 09 19:10:14.267464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcc305-f523-4611-874d-db5fab39e1e5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:14.292773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 741/1501] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:10:14 2022] PUT /identity/v3/projects/e43c25434e5f4929b87abde75ea86e8f/users/f8e6315af02143cb91ec6c80129ed96e/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:10:14.305459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca3d1c-34a2-4777-8f63-a7fe1ba66377 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:14.305748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca3d1c-34a2-4777-8f63-a7fe1ba66377 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:14.306021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca3d1c-34a2-4777-8f63-a7fe1ba66377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:14.362347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-03ca3d1c-34a2-4777-8f63-a7fe1ba66377 None None] MFA Rules not processed for user `f8e6315af02143cb91ec6c80129ed96e`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:10:14.363474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-03ca3d1c-34a2-4777-8f63-a7fe1ba66377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:14.441424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 761/1502] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:10:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:10:14.455441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c0fd-b495-4894-80c0-d6db7a168efe None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:14.455727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c0fd-b495-4894-80c0-d6db7a168efe None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:14.455999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c0fd-b495-4894-80c0-d6db7a168efe None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:14.457549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 742/1503] 10.222.0.70 () {54 vars in 964 bytes} [Tue Aug 9 19:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:10:14.462334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6a7b4-71c6-4ff1-a08f-ebfa039df43b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:14.462617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6a7b4-71c6-4ff1-a08f-ebfa039df43b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:14.462911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6a7b4-71c6-4ff1-a08f-ebfa039df43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:14.495001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:14.538969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-07b6a7b4-71c6-4ff1-a08f-ebfa039df43b None None] MFA Rules not processed for user `00be9b6d551a4ba39718a44101ecf9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:10:14.540163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07b6a7b4-71c6-4ff1-a08f-ebfa039df43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:14.613677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 762/1504] 10.222.0.70 () {58 vars in 1061 bytes} [Tue Aug 9 19:10:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:10:14.621414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c0fd-b495-4894-80c0-d6db7a168efe None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:14.622930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3f79c0fd-b495-4894-80c0-d6db7a168efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:14.673297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:14.731547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c0fd-b495-4894-80c0-d6db7a168efe None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:14.807194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:14.807885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:14.808170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:14.808467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:14.810088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:14.909482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:14.910915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:14.914681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:14.930461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 743/1505] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:10:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:10:14.975355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e0c4c-878d-47e2-be03-e83520bbaa7c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:14.975641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e0c4c-878d-47e2-be03-e83520bbaa7c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:14.975911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e0c4c-878d-47e2-be03-e83520bbaa7c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:14.977198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 763/1506] 10.222.0.70 () {52 vars in 830 bytes} [Tue Aug 9 19:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:10:14.980673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:14.981887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:15.083683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac58f4-3612-4e12-99c0-111eed2c628e service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:15.177298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:15.177935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:15.178402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:15.178674 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] PATH_INFO: `/v3/users/f8e6315af02143cb91ec6c80129ed96e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:15.192280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e6315af02143cb91ec6c80129ed96e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:15.193344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:10:15.193344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Aug 09 19:10:15.193344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: Aug 09 19:10:15.194545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:15.204902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:15.264304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e6315af02143cb91ec6c80129ed96e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:15.273231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e6315af02143cb91ec6c80129ed96e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:15.287713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e6315af02143cb91ec6c80129ed96e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:15.288253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e6315af02143cb91ec6c80129ed96e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:15.288706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e6315af02143cb91ec6c80129ed96e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:15.317139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 744/1507] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:10:14 2022] DELETE /identity/v3/users/f8e6315af02143cb91ec6c80129ed96e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:10:15.321502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcb5d8-b8b7-472c-89b7-a7d1c89d266c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:15.321990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcb5d8-b8b7-472c-89b7-a7d1c89d266c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:15.322089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcb5d8-b8b7-472c-89b7-a7d1c89d266c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:15.377996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6bbcb5d8-b8b7-472c-89b7-a7d1c89d266c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:10:15.379199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6bbcb5d8-b8b7-472c-89b7-a7d1c89d266c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:15.455946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:15.469727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 764/1508] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:10:15.480057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:15.481296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:15.589970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3e55-7a09-413a-b980-8ffe7f1e19bc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:15.644990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:15.666638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ec5e5-4bf0-47cb-b1dd-9f4832ac414f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:15.667247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec5e5-4bf0-47cb-b1dd-9f4832ac414f service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:15.667526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec5e5-4bf0-47cb-b1dd-9f4832ac414f service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:15.667798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec5e5-4bf0-47cb-b1dd-9f4832ac414f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:15.669446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a21ec5e5-4bf0-47cb-b1dd-9f4832ac414f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:15.779437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ec5e5-4bf0-47cb-b1dd-9f4832ac414f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:15.780809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ec5e5-4bf0-47cb-b1dd-9f4832ac414f service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:15.800068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 745/1509] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:10:16.165554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcb5d8-b8b7-472c-89b7-a7d1c89d266c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:10:16.166781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6bbcb5d8-b8b7-472c-89b7-a7d1c89d266c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:10:16.178627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcb5d8-b8b7-472c-89b7-a7d1c89d266c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:10:16.274194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:10:16.274812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:10:16.275083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:10:16.275348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] PATH_INFO: `/v3/projects/e43c25434e5f4929b87abde75ea86e8f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:10:16.285243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43c25434e5f4929b87abde75ea86e8f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:10:16.286304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:10:16.286304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Aug 09 19:10:16.286304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: Aug 09 19:10:16.287462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:10:16.346173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43c25434e5f4929b87abde75ea86e8f', 'request_id': 'req-5a39752d-8418-4820-9981-15a0ab186f8b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:16.352019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43c25434e5f4929b87abde75ea86e8f', 'request_id': 'req-5a39752d-8418-4820-9981-15a0ab186f8b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:16.369195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43c25434e5f4929b87abde75ea86e8f', 'request_id': 'req-5a39752d-8418-4820-9981-15a0ab186f8b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:16.408103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] The token cache is being invalidate because project e43c25434e5f4929b87abde75ea86e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:10:16.408565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:16.409202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5a39752d-8418-4820-9981-15a0ab186f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:10:16.410668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 765/1510] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:10:16 2022] DELETE /identity/v3/projects/e43c25434e5f4929b87abde75ea86e8f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:10:25.797559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=62 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:10:26.405821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:11:47.166960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fd5fa-8aed-4d15-aa50-28fd25e1ade2 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:47.167591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fd5fa-8aed-4d15-aa50-28fd25e1ade2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:47.167591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fd5fa-8aed-4d15-aa50-28fd25e1ade2 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:47.168872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:11:47.173794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfee05-3f71-4bd3-aa07-8bb009830d0d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:47.174258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfee05-3f71-4bd3-aa07-8bb009830d0d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:47.174361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfee05-3f71-4bd3-aa07-8bb009830d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:47.256347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0dbfee05-3f71-4bd3-aa07-8bb009830d0d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:11:47.257577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0dbfee05-3f71-4bd3-aa07-8bb009830d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:11:47.360331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:11:47.514679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fd5fa-8aed-4d15-aa50-28fd25e1ade2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:11:47.516816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e68fd5fa-8aed-4d15-aa50-28fd25e1ade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:11:47.639816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fd5fa-8aed-4d15-aa50-28fd25e1ade2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:11:47.645443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b88a7-7aa2-4fd0-9674-041742b763c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:11:47.646051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b88a7-7aa2-4fd0-9674-041742b763c4 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:47.646330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b88a7-7aa2-4fd0-9674-041742b763c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:47.646603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b88a7-7aa2-4fd0-9674-041742b763c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:47.655132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b88a7-7aa2-4fd0-9674-041742b763c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:11:47.656745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b88a7-7aa2-4fd0-9674-041742b763c4 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:11:47.666253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 747/1513] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:11:48.501744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3fc8-5918-44e7-9fcc-b6ea55924b73 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:48.502298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3fc8-5918-44e7-9fcc-b6ea55924b73 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:48.502298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3fc8-5918-44e7-9fcc-b6ea55924b73 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:48.503670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:11:48.508591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241d98-c3b8-4a01-942a-0d59284f72c8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:48.509141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241d98-c3b8-4a01-942a-0d59284f72c8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:48.509141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241d98-c3b8-4a01-942a-0d59284f72c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:48.528771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-60241d98-c3b8-4a01-942a-0d59284f72c8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:11:48.529947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-60241d98-c3b8-4a01-942a-0d59284f72c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:11:48.537630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:11:48.689925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb3fc8-5918-44e7-9fcc-b6ea55924b73 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:11:48.704301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb3fc8-5918-44e7-9fcc-b6ea55924b73 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:11:48.709763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175398d-29c5-41a9-aa3b-bdabebc3a050 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:11:48.710364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175398d-29c5-41a9-aa3b-bdabebc3a050 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:48.710647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175398d-29c5-41a9-aa3b-bdabebc3a050 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:48.710922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175398d-29c5-41a9-aa3b-bdabebc3a050 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:48.719980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175398d-29c5-41a9-aa3b-bdabebc3a050 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:11:48.721613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175398d-29c5-41a9-aa3b-bdabebc3a050 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:11:48.731730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 768/1516] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:11:49.371176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428743-fe2a-4ce9-9034-0f2f212a50c6 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:49.371442 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428743-fe2a-4ce9-9034-0f2f212a50c6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:49.371699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428743-fe2a-4ce9-9034-0f2f212a50c6 None None] PATH_INFO: `/v3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:49.372995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 749/1517] 10.222.0.60 () {54 vars in 925 bytes} [Tue Aug 9 19:11:49 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:11:49.389701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3221e0a-726c-4d34-9125-1b4becddc88e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:49.390303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3221e0a-726c-4d34-9125-1b4becddc88e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:49.390461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3221e0a-726c-4d34-9125-1b4becddc88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:49.449420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a3221e0a-726c-4d34-9125-1b4becddc88e None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:11:49.450605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a3221e0a-726c-4d34-9125-1b4becddc88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:11:49.467969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:11:49.522579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 769/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Tue Aug 9 19:11:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:11:53.346247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d258418-c90e-4954-8bec-978539d232d2 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:53.346898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d258418-c90e-4954-8bec-978539d232d2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:53.346898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d258418-c90e-4954-8bec-978539d232d2 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:53.348088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 750/1519] 10.222.0.70 () {54 vars in 961 bytes} [Tue Aug 9 19:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:11:53.353380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59214a-eb98-46da-8588-4d8ebe45c63b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:53.353779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59214a-eb98-46da-8588-4d8ebe45c63b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:53.353937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59214a-eb98-46da-8588-4d8ebe45c63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:53.372904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1c59214a-eb98-46da-8588-4d8ebe45c63b None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:11:53.374098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1c59214a-eb98-46da-8588-4d8ebe45c63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:11:53.382935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 770/1520] 10.222.0.70 () {58 vars in 1058 bytes} [Tue Aug 9 19:11:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:11:53.392766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d258418-c90e-4954-8bec-978539d232d2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:11:53.406844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d258418-c90e-4954-8bec-978539d232d2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:11:53.411779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e4dc0-785e-42b6-ad0c-9d732665e7ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:11:53.412389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e4dc0-785e-42b6-ad0c-9d732665e7ec service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:53.412672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e4dc0-785e-42b6-ad0c-9d732665e7ec service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:53.412943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e4dc0-785e-42b6-ad0c-9d732665e7ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:53.421213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e4dc0-785e-42b6-ad0c-9d732665e7ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:11:53.422800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e4dc0-785e-42b6-ad0c-9d732665e7ec service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:11:53.432671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 751/1521] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:11:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:11:59.971737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c416f-1eac-4e49-987b-e63f5ff8d612 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:59.972180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c416f-1eac-4e49-987b-e63f5ff8d612 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:59.972274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c416f-1eac-4e49-987b-e63f5ff8d612 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:59.973667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:11:59.978516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee64e2-0536-4e80-b28d-ecf4311f0a9c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:11:59.978985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee64e2-0536-4e80-b28d-ecf4311f0a9c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:11:59.979157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee64e2-0536-4e80-b28d-ecf4311f0a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:11:59.997882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b8ee64e2-0536-4e80-b28d-ecf4311f0a9c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:11:59.999171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b8ee64e2-0536-4e80-b28d-ecf4311f0a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:00.006980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:12:00.166879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c416f-1eac-4e49-987b-e63f5ff8d612 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:12:00.181956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c416f-1eac-4e49-987b-e63f5ff8d612 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:12:00.188035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763f306-0383-4c7d-829a-b8b71084f666 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:12:00.188658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f306-0383-4c7d-829a-b8b71084f666 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:00.188937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f306-0383-4c7d-829a-b8b71084f666 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:00.189209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f306-0383-4c7d-829a-b8b71084f666 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:00.197256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763f306-0383-4c7d-829a-b8b71084f666 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:12:00.199058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763f306-0383-4c7d-829a-b8b71084f666 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:12:00.208400 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 772/1524] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:12:01.023879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce05d4-3418-4bb6-8a76-7841a7f72668 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:01.024489 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce05d4-3418-4bb6-8a76-7841a7f72668 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:01.024489 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce05d4-3418-4bb6-8a76-7841a7f72668 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:01.025867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 19:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:12:01.030715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c8734-2041-4dbb-b84b-54bfd27b713a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:01.031164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c8734-2041-4dbb-b84b-54bfd27b713a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:01.031260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c8734-2041-4dbb-b84b-54bfd27b713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:01.052556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-838c8734-2041-4dbb-b84b-54bfd27b713a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:12:01.053753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-838c8734-2041-4dbb-b84b-54bfd27b713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:01.062720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 19:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:12:01.215188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce05d4-3418-4bb6-8a76-7841a7f72668 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:12:01.234872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce05d4-3418-4bb6-8a76-7841a7f72668 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:12:01.241011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938b0e1-977d-4cf9-901c-145b6fa00625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:12:01.241608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938b0e1-977d-4cf9-901c-145b6fa00625 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:01.241881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938b0e1-977d-4cf9-901c-145b6fa00625 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:01.242147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938b0e1-977d-4cf9-901c-145b6fa00625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:01.250412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938b0e1-977d-4cf9-901c-145b6fa00625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:12:01.251966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938b0e1-977d-4cf9-901c-145b6fa00625 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:12:01.261299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 754/1527] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:12:06.281817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df99a-ea65-4659-b967-34c561e70531 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:06.282363 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df99a-ea65-4659-b967-34c561e70531 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:06.282363 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df99a-ea65-4659-b967-34c561e70531 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:06.283621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:12:06.288922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d13ea-ba99-47ca-b679-53bd0c5707d6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:06.289323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d13ea-ba99-47ca-b679-53bd0c5707d6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:06.289465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d13ea-ba99-47ca-b679-53bd0c5707d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:06.310251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0f4d13ea-ba99-47ca-b679-53bd0c5707d6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:12:06.311438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0f4d13ea-ba99-47ca-b679-53bd0c5707d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:06.319941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:12:06.395693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df99a-ea65-4659-b967-34c561e70531 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:12:06.397403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e40df99a-ea65-4659-b967-34c561e70531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:06.473082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df99a-ea65-4659-b967-34c561e70531 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:12:06.476562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391809b-134b-4a65-9962-922331e04df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:12:06.477148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391809b-134b-4a65-9962-922331e04df5 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:06.477433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391809b-134b-4a65-9962-922331e04df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:06.477686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391809b-134b-4a65-9962-922331e04df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:06.486675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391809b-134b-4a65-9962-922331e04df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:12:06.488090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391809b-134b-4a65-9962-922331e04df5 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:12:06.498168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 775/1530] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:12:08.541201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b56e3-a9c1-4390-8865-eccf497c27af None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:08.541750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b56e3-a9c1-4390-8865-eccf497c27af None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:08.541750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b56e3-a9c1-4390-8865-eccf497c27af None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:08.543018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:12:08.548241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835caaa2-f324-41ef-8209-eea72c8793bb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:08.548751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835caaa2-f324-41ef-8209-eea72c8793bb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:08.548845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835caaa2-f324-41ef-8209-eea72c8793bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:08.569412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-835caaa2-f324-41ef-8209-eea72c8793bb None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:12:08.570615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-835caaa2-f324-41ef-8209-eea72c8793bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:08.579299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:12:08.614016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b56e3-a9c1-4390-8865-eccf497c27af None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:12:08.625377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b56e3-a9c1-4390-8865-eccf497c27af None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:12:08.628850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c7992-319d-4b61-b232-0e7d4a510f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:12:08.629439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c7992-319d-4b61-b232-0e7d4a510f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:08.629714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c7992-319d-4b61-b232-0e7d4a510f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:08.629984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c7992-319d-4b61-b232-0e7d4a510f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:08.639222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c7992-319d-4b61-b232-0e7d4a510f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:12:08.640693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c7992-319d-4b61-b232-0e7d4a510f6a service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:12:08.649912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 757/1533] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:12:12.221522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87eb9-1e35-4573-981f-04b2eca908e3 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:12.222089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87eb9-1e35-4573-981f-04b2eca908e3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:12.222089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87eb9-1e35-4573-981f-04b2eca908e3 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:12.223353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:12:12.228350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839a56e-f72c-4216-b653-c8a59e617c21 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:12.228822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839a56e-f72c-4216-b653-c8a59e617c21 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:12.228924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839a56e-f72c-4216-b653-c8a59e617c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:12.250012 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6839a56e-f72c-4216-b653-c8a59e617c21 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:12:12.252045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6839a56e-f72c-4216-b653-c8a59e617c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:12.261810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:12:12.581374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87eb9-1e35-4573-981f-04b2eca908e3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:12:12.583098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ea87eb9-1e35-4573-981f-04b2eca908e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:12.659448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:12:12.672884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87eb9-1e35-4573-981f-04b2eca908e3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:12:12.676372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f93688-1103-4bda-8aa7-8fb45fba1e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:12:12.677334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f93688-1103-4bda-8aa7-8fb45fba1e37 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:12.677611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f93688-1103-4bda-8aa7-8fb45fba1e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:12.677883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f93688-1103-4bda-8aa7-8fb45fba1e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:12.687160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f93688-1103-4bda-8aa7-8fb45fba1e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:12:12.688616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f93688-1103-4bda-8aa7-8fb45fba1e37 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:12:12.697689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 778/1536] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:12:14.777273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a6e6c-eac9-4aac-9319-836c664f25cc None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:14.778075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a6e6c-eac9-4aac-9319-836c664f25cc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:14.778075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a6e6c-eac9-4aac-9319-836c664f25cc None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:14.779103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 19:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:12:14.784184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32be92-1d0d-4d00-b613-31068358f1a9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:14.784647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32be92-1d0d-4d00-b613-31068358f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:14.784753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32be92-1d0d-4d00-b613-31068358f1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:14.804888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cd32be92-1d0d-4d00-b613-31068358f1a9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:12:14.806092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cd32be92-1d0d-4d00-b613-31068358f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:14.814810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 19:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:12:15.115203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a6e6c-eac9-4aac-9319-836c664f25cc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:12:15.116885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8c3a6e6c-eac9-4aac-9319-836c664f25cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:12:15.130454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a6e6c-eac9-4aac-9319-836c664f25cc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:12:15.134237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ec05d-eb19-4136-a043-c3098544e315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:12:15.134897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ec05d-eb19-4136-a043-c3098544e315 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:12:15.135167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ec05d-eb19-4136-a043-c3098544e315 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:12:15.135523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ec05d-eb19-4136-a043-c3098544e315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:12:15.144524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ec05d-eb19-4136-a043-c3098544e315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:12:15.145919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ec05d-eb19-4136-a043-c3098544e315 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:12:15.155021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 760/1539] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:12:24.799795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=11 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:12:25.151027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:42.537602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ea1db-d573-4729-a93b-b1ef24d992bb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.540774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ea1db-d573-4729-a93b-b1ef24d992bb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.540774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ea1db-d573-4729-a93b-b1ef24d992bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.565417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-026ea1db-d573-4729-a93b-b1ef24d992bb None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:42.566746 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-026ea1db-d573-4729-a93b-b1ef24d992bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:42.577799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 780/1540] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:42.589458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d764c8-07b0-4ec6-a73b-e03589458e99 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.590348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d764c8-07b0-4ec6-a73b-e03589458e99 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.590348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d764c8-07b0-4ec6-a73b-e03589458e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.638418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d5d764c8-07b0-4ec6-a73b-e03589458e99 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:42.640393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d5d764c8-07b0-4ec6-a73b-e03589458e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:42.646474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a35b-17f5-4339-b7bb-c5db1c31c227 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.647137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a35b-17f5-4339-b7bb-c5db1c31c227 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.647625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a35b-17f5-4339-b7bb-c5db1c31c227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.666355 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 761/1541] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:42.670679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d764c8-07b0-4ec6-a73b-e03589458e99 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:42.682171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3e70a35b-17f5-4339-b7bb-c5db1c31c227 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:42.683560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3e70a35b-17f5-4339-b7bb-c5db1c31c227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:42.694491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d764c8-07b0-4ec6-a73b-e03589458e99 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:42.720073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70e19f-fd84-4c05-acc4-f33ae6908f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:42.721888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70e19f-fd84-4c05-acc4-f33ae6908f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.722686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70e19f-fd84-4c05-acc4-f33ae6908f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.723375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70e19f-fd84-4c05-acc4-f33ae6908f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.723817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 781/1542] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:42.729265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70e19f-fd84-4c05-acc4-f33ae6908f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:42.730159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70e19f-fd84-4c05-acc4-f33ae6908f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:42.733069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-bb70e19f-fd84-4c05-acc4-f33ae6908f18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:13:42.733069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Aug 09 19:13:42.733069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: Aug 09 19:13:42.734067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d743d-43a7-4323-82eb-e13fef133fde None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.734407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d743d-43a7-4323-82eb-e13fef133fde None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.734745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d743d-43a7-4323-82eb-e13fef133fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.736140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70e19f-fd84-4c05-acc4-f33ae6908f18 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:42.769356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 762/1543] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:13:42.771919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bc26b-2741-4246-8f2e-7acbbc23def5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.772713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bc26b-2741-4246-8f2e-7acbbc23def5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.773432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bc26b-2741-4246-8f2e-7acbbc23def5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.777128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1f6d743d-43a7-4323-82eb-e13fef133fde None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:42.778595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1f6d743d-43a7-4323-82eb-e13fef133fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:42.807994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 782/1544] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:42.809223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d743d-43a7-4323-82eb-e13fef133fde None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:42.829243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d743d-43a7-4323-82eb-e13fef133fde None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:42.835840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f66bc26b-2741-4246-8f2e-7acbbc23def5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:42.839824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f66bc26b-2741-4246-8f2e-7acbbc23def5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:42.853310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c6535-be9b-4c05-a5cf-09317a22d758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:42.853975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6535-be9b-4c05-a5cf-09317a22d758 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.854252 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6535-be9b-4c05-a5cf-09317a22d758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.854532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6535-be9b-4c05-a5cf-09317a22d758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.855317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c6535-be9b-4c05-a5cf-09317a22d758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:42.856370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-ab9c6535-be9b-4c05-a5cf-09317a22d758 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:13:42.856370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Aug 09 19:13:42.856370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: Aug 09 19:13:42.857605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c6535-be9b-4c05-a5cf-09317a22d758 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:42.866972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 763/1545] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:42.869392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bc26b-2741-4246-8f2e-7acbbc23def5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:42.903894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bc26b-2741-4246-8f2e-7acbbc23def5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:42.920367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcf416-cbb4-454c-a7a2-6f03d28e7689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:42.920973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcf416-cbb4-454c-a7a2-6f03d28e7689 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.921275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcf416-cbb4-454c-a7a2-6f03d28e7689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.921552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcf416-cbb4-454c-a7a2-6f03d28e7689 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.922209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfcf416-cbb4-454c-a7a2-6f03d28e7689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:42.922508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfcf416-cbb4-454c-a7a2-6f03d28e7689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:42.924296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfcf416-cbb4-454c-a7a2-6f03d28e7689 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:42.943652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 783/1546] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:42 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:42.945734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f21bfa-1f72-45d2-b403-f5e3c5e8640a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.945734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f21bfa-1f72-45d2-b403-f5e3c5e8640a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.946552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f21bfa-1f72-45d2-b403-f5e3c5e8640a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.955918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 764/1547] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:42.958492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a02d-9087-4c95-b674-1048335cc4a3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:42.958807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a02d-9087-4c95-b674-1048335cc4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:42.959096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9a02d-9087-4c95-b674-1048335cc4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:42.983602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-86f21bfa-1f72-45d2-b403-f5e3c5e8640a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:42.985764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-86f21bfa-1f72-45d2-b403-f5e3c5e8640a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.007257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a7f9a02d-9087-4c95-b674-1048335cc4a3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.009098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a7f9a02d-9087-4c95-b674-1048335cc4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.097758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 784/1548] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.099167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b233ef-8404-4799-a5bf-3cfa20a7b4a0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.099448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b233ef-8404-4799-a5bf-3cfa20a7b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.099751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b233ef-8404-4799-a5bf-3cfa20a7b4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.124366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f2b233ef-8404-4799-a5bf-3cfa20a7b4a0 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.125592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f2b233ef-8404-4799-a5bf-3cfa20a7b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.130722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 765/1549] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.132635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a02d-9087-4c95-b674-1048335cc4a3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.147175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 785/1550] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.148033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b233ef-8404-4799-a5bf-3cfa20a7b4a0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.152875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9a02d-9087-4c95-b674-1048335cc4a3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.161860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4ff27-78d1-46d4-a0f7-f9303ea395b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.163514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4ff27-78d1-46d4-a0f7-f9303ea395b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.164306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4ff27-78d1-46d4-a0f7-f9303ea395b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.165718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4ff27-78d1-46d4-a0f7-f9303ea395b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.167141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4ff27-78d1-46d4-a0f7-f9303ea395b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.169564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b233ef-8404-4799-a5bf-3cfa20a7b4a0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.169935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-ebe4ff27-78d1-46d4-a0f7-f9303ea395b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:13:43.169935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Aug 09 19:13:43.169935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: Aug 09 19:13:43.173315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4ff27-78d1-46d4-a0f7-f9303ea395b5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.175663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de33b2-7051-4974-b600-f159e824aec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.176278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de33b2-7051-4974-b600-f159e824aec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.177256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de33b2-7051-4974-b600-f159e824aec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.177596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de33b2-7051-4974-b600-f159e824aec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.178288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de33b2-7051-4974-b600-f159e824aec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.180860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de33b2-7051-4974-b600-f159e824aec6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.210908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 766/1551] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/users => generated 287 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.213059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87b30a-d66b-4ae1-aee4-f1cd83dd78f6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.213435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87b30a-d66b-4ae1-aee4-f1cd83dd78f6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.213713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87b30a-d66b-4ae1-aee4-f1cd83dd78f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.231994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 786/1552] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/projects => generated 391 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.231994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b60c-ac14-40b1-b349-530ca33a1c0a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.231994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b60c-ac14-40b1-b349-530ca33a1c0a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.231994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031b60c-ac14-40b1-b349-530ca33a1c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.234963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4d87b30a-d66b-4ae1-aee4-f1cd83dd78f6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.236220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4d87b30a-d66b-4ae1-aee4-f1cd83dd78f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.266103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3031b60c-ac14-40b1-b349-530ca33a1c0a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.267326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3031b60c-ac14-40b1-b349-530ca33a1c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.301045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 767/1553] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.301890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87b30a-d66b-4ae1-aee4-f1cd83dd78f6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.312088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87b30a-d66b-4ae1-aee4-f1cd83dd78f6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.317220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3027f-9d7a-4ee3-9e69-ef9184598632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.317812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3027f-9d7a-4ee3-9e69-ef9184598632 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.318087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3027f-9d7a-4ee3-9e69-ef9184598632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.318354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3027f-9d7a-4ee3-9e69-ef9184598632 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.319033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3027f-9d7a-4ee3-9e69-ef9184598632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:43.319594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3027f-9d7a-4ee3-9e69-ef9184598632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.321524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3027f-9d7a-4ee3-9e69-ef9184598632 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.331341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 787/1554] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.332697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2ef18-c9c1-4378-a142-864bc9a17c4a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.332983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2ef18-c9c1-4378-a142-864bc9a17c4a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.333258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2ef18-c9c1-4378-a142-864bc9a17c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.340111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 768/1555] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:43.341728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba254719-edd5-4673-8e14-5afa85a2149e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.342834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba254719-edd5-4673-8e14-5afa85a2149e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.343118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba254719-edd5-4673-8e14-5afa85a2149e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.353503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-48e2ef18-c9c1-4378-a142-864bc9a17c4a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.354738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-48e2ef18-c9c1-4378-a142-864bc9a17c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.362604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 788/1556] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.363115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ba254719-edd5-4673-8e14-5afa85a2149e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.363454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2ef18-c9c1-4378-a142-864bc9a17c4a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.364271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ba254719-edd5-4673-8e14-5afa85a2149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.372918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 769/1557] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.373693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba254719-edd5-4673-8e14-5afa85a2149e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.373888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2ef18-c9c1-4378-a142-864bc9a17c4a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.380559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ff40c-cfdf-45b5-a219-8ce87cf9bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.381172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff40c-cfdf-45b5-a219-8ce87cf9bfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.381452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff40c-cfdf-45b5-a219-8ce87cf9bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.381724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ff40c-cfdf-45b5-a219-8ce87cf9bfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.382372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ff40c-cfdf-45b5-a219-8ce87cf9bfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.383284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba254719-edd5-4673-8e14-5afa85a2149e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.384645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ff40c-cfdf-45b5-a219-8ce87cf9bfb8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.389782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48092a10-b374-40a4-8f47-3c424b4b747a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.390105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092a10-b374-40a4-8f47-3c424b4b747a admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.390302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092a10-b374-40a4-8f47-3c424b4b747a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.390622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48092a10-b374-40a4-8f47-3c424b4b747a admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.391347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48092a10-b374-40a4-8f47-3c424b4b747a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.393320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48092a10-b374-40a4-8f47-3c424b4b747a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.396310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 789/1558] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:43.397695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68339-b2fb-4eb9-9872-1e1b5cbdf4a2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.397980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68339-b2fb-4eb9-9872-1e1b5cbdf4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.398256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68339-b2fb-4eb9-9872-1e1b5cbdf4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.407968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:43.418368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 770/1559] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/users => generated 296 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.419882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6138-ac35-4c85-934f-ca1c89221d15 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.420222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6138-ac35-4c85-934f-ca1c89221d15 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.420605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6138-ac35-4c85-934f-ca1c89221d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.420916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-21d68339-b2fb-4eb9-9872-1e1b5cbdf4a2 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.422966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-21d68339-b2fb-4eb9-9872-1e1b5cbdf4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.432953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 790/1560] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.433792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d68339-b2fb-4eb9-9872-1e1b5cbdf4a2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.435887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:43.441936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6deb6138-ac35-4c85-934f-ca1c89221d15 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.443738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6deb6138-ac35-4c85-934f-ca1c89221d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.444521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d68339-b2fb-4eb9-9872-1e1b5cbdf4a2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.449074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.449661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.449938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.450207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.450838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:43.451134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.452030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 771/1561] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.452836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb6138-ac35-4c85-934f-ca1c89221d15 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.453026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.465407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb6138-ac35-4c85-934f-ca1c89221d15 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.465670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 791/1562] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:43.466309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.474184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4412bb-3142-4cd3-9b1d-2a1331783e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.474790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4412bb-3142-4cd3-9b1d-2a1331783e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.475189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4412bb-3142-4cd3-9b1d-2a1331783e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.475344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4412bb-3142-4cd3-9b1d-2a1331783e3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.475968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4412bb-3142-4cd3-9b1d-2a1331783e3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.478971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4412bb-3142-4cd3-9b1d-2a1331783e3b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.480934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10528f-1ac8-455f-8fe2-2f94431ffcdf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.485416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60548a-45e2-4db5-93fb-568c001e6e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.485999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60548a-45e2-4db5-93fb-568c001e6e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.486272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60548a-45e2-4db5-93fb-568c001e6e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.486539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60548a-45e2-4db5-93fb-568c001e6e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.487166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60548a-45e2-4db5-93fb-568c001e6e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:43.487492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60548a-45e2-4db5-93fb-568c001e6e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.489159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60548a-45e2-4db5-93fb-568c001e6e94 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.501652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 792/1563] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:43.502842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e6b82-b704-4018-90cb-aaa7fe206301 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.503120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e6b82-b704-4018-90cb-aaa7fe206301 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.503497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e6b82-b704-4018-90cb-aaa7fe206301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.511743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 772/1564] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.511743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4412bb-3142-4cd3-9b1d-2a1331783e3b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.526451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a51e6b82-b704-4018-90cb-aaa7fe206301 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.527716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a51e6b82-b704-4018-90cb-aaa7fe206301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.572646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4412bb-3142-4cd3-9b1d-2a1331783e3b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.577337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c05a1-7d4a-4e51-b8bc-ecddeaceafae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.577936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c05a1-7d4a-4e51-b8bc-ecddeaceafae admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.578210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c05a1-7d4a-4e51-b8bc-ecddeaceafae admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.578474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c05a1-7d4a-4e51-b8bc-ecddeaceafae admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.579112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c05a1-7d4a-4e51-b8bc-ecddeaceafae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.580714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c05a1-7d4a-4e51-b8bc-ecddeaceafae admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.588694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 793/1565] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.589320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e6b82-b704-4018-90cb-aaa7fe206301 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.594475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 773/1566] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:43.594475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00930-c200-4cc4-b4d7-389cd89dd4d3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.594475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00930-c200-4cc4-b4d7-389cd89dd4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.594475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad00930-c200-4cc4-b4d7-389cd89dd4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.598957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e6b82-b704-4018-90cb-aaa7fe206301 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.604126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf0e1e-3ea2-48d0-9f53-8150e4d10f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.604751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf0e1e-3ea2-48d0-9f53-8150e4d10f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.605031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf0e1e-3ea2-48d0-9f53-8150e4d10f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.605300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf0e1e-3ea2-48d0-9f53-8150e4d10f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.605977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf0e1e-3ea2-48d0-9f53-8150e4d10f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.607568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf0e1e-3ea2-48d0-9f53-8150e4d10f8d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.613147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7ad00930-c200-4cc4-b4d7-389cd89dd4d3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:43.614439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7ad00930-c200-4cc4-b4d7-389cd89dd4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:43.617582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 794/1567] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:43.618591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf0e1e-3ea2-48d0-9f53-8150e4d10f8d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.625653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 774/1568] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:43.626444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad00930-c200-4cc4-b4d7-389cd89dd4d3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.628759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf0e1e-3ea2-48d0-9f53-8150e4d10f8d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.634518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.634991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.635184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.635446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.636731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:43.637057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.640293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.640904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad00930-c200-4cc4-b4d7-389cd89dd4d3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.646165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502fe5-d781-4569-99b0-6430076038ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.646770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502fe5-d781-4569-99b0-6430076038ee admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.647467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502fe5-d781-4569-99b0-6430076038ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.647467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502fe5-d781-4569-99b0-6430076038ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.647930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac502fe5-d781-4569-99b0-6430076038ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.651030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac502fe5-d781-4569-99b0-6430076038ee admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.653708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 795/1569] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:43.654495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.668835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacf367-7dac-4bfe-a650-f2c27d277ac4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.673741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef95e5-250b-4b05-8f0a-95bbe50f923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.674375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef95e5-250b-4b05-8f0a-95bbe50f923d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.674641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef95e5-250b-4b05-8f0a-95bbe50f923d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.674986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef95e5-250b-4b05-8f0a-95bbe50f923d admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.675875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef95e5-250b-4b05-8f0a-95bbe50f923d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.677870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef95e5-250b-4b05-8f0a-95bbe50f923d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.687823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 775/1570] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.687823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502fe5-d781-4569-99b0-6430076038ee admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.706110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 796/1571] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.707340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef95e5-250b-4b05-8f0a-95bbe50f923d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.751499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502fe5-d781-4569-99b0-6430076038ee admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.769512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef95e5-250b-4b05-8f0a-95bbe50f923d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.774620 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.775447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.775913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.776196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.776878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:43.777185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.778823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.791638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 797/1572] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:43.792424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.800030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:43.804820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb2b77-0a61-4840-8513-a807a4b59459 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.810404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.810679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409811e-b109-4271-ae00-1502396f865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.811083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.811359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.811485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409811e-b109-4271-ae00-1502396f865a admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.811609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] PATH_INFO: `/v3/projects/1a0b9850f88142be9093b65265fbd8f4/users/1bb6b95759d642fe9dbc24793089a70f/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.811748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409811e-b109-4271-ae00-1502396f865a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.812025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409811e-b109-4271-ae00-1502396f865a admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.812707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409811e-b109-4271-ae00-1502396f865a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.814651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409811e-b109-4271-ae00-1502396f865a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.826657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0b9850f88142be9093b65265fbd8f4, user_id=1bb6b95759d642fe9dbc24793089a70f, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.827791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:13:43.827791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Aug 09 19:13:43.827791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: Aug 09 19:13:43.829869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.833007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:43.841315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 776/1573] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.841315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409811e-b109-4271-ae00-1502396f865a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.847368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 798/1574] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:43 2022] PUT /identity/v3/projects/1a0b9850f88142be9093b65265fbd8f4/users/1bb6b95759d642fe9dbc24793089a70f/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:43.848174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.850918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409811e-b109-4271-ae00-1502396f865a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.911257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7969fa-5e8f-4ba1-ad69-540e32546723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.911873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7969fa-5e8f-4ba1-ad69-540e32546723 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.912006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce9fc4-03df-4f02-81e3-05416c558828 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.912211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7969fa-5e8f-4ba1-ad69-540e32546723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.912473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7969fa-5e8f-4ba1-ad69-540e32546723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.913127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7969fa-5e8f-4ba1-ad69-540e32546723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.915546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7969fa-5e8f-4ba1-ad69-540e32546723 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.917855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.918462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.918734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.919029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.919663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:43.919958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.921568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.925818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 777/1575] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:43.926713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7969fa-5e8f-4ba1-ad69-540e32546723 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.936473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 799/1576] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:43.936473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.943213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7969fa-5e8f-4ba1-ad69-540e32546723 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.944553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d47b7a-e9a1-4cee-adfd-da750c0f9ac3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:43.948368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4ebb4-f84e-4a80-9206-e5e46b69201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.949001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4ebb4-f84e-4a80-9206-e5e46b69201e admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.949304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4ebb4-f84e-4a80-9206-e5e46b69201e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.949441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91233c-2c56-4948-a5a5-20d02f121720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:43.949559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4ebb4-f84e-4a80-9206-e5e46b69201e admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.949970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91233c-2c56-4948-a5a5-20d02f121720 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:43.950188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4ebb4-f84e-4a80-9206-e5e46b69201e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.950304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91233c-2c56-4948-a5a5-20d02f121720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:43.950524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91233c-2c56-4948-a5a5-20d02f121720 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:43.951164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91233c-2c56-4948-a5a5-20d02f121720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:43.952100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4ebb4-f84e-4a80-9206-e5e46b69201e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.953141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91233c-2c56-4948-a5a5-20d02f121720 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:43.975929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 800/1577] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/users => generated 291 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.979765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91233c-2c56-4948-a5a5-20d02f121720 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.987663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 778/1578] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:43.987663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4ebb4-f84e-4a80-9206-e5e46b69201e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:43.987898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91233c-2c56-4948-a5a5-20d02f121720 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.045423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94763a-d3fd-45ed-959d-9cce0aaaf23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.046034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94763a-d3fd-45ed-959d-9cce0aaaf23e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.046311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94763a-d3fd-45ed-959d-9cce0aaaf23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.046582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94763a-d3fd-45ed-959d-9cce0aaaf23e admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.047232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94763a-d3fd-45ed-959d-9cce0aaaf23e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.049185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94763a-d3fd-45ed-959d-9cce0aaaf23e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.049332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4ebb4-f84e-4a80-9206-e5e46b69201e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.054170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47622e1-fe8f-484c-94ea-14c476538934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.054769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47622e1-fe8f-484c-94ea-14c476538934 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.055070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47622e1-fe8f-484c-94ea-14c476538934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.055340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47622e1-fe8f-484c-94ea-14c476538934 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.055999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47622e1-fe8f-484c-94ea-14c476538934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.058198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47622e1-fe8f-484c-94ea-14c476538934 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.074790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 801/1579] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:44.074790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94763a-d3fd-45ed-959d-9cce0aaaf23e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.081907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94763a-d3fd-45ed-959d-9cce0aaaf23e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.087469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d242d-8048-426f-851f-00d111033d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.088111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d242d-8048-426f-851f-00d111033d3a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.088387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d242d-8048-426f-851f-00d111033d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.088693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d242d-8048-426f-851f-00d111033d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.089325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d242d-8048-426f-851f-00d111033d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.090910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d242d-8048-426f-851f-00d111033d3a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.092638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 779/1580] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:43 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:44.097077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d917e-1b82-47ce-82ac-bfe364802ede None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.097365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d917e-1b82-47ce-82ac-bfe364802ede None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.097638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d917e-1b82-47ce-82ac-bfe364802ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.102625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 802/1581] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.103496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d242d-8048-426f-851f-00d111033d3a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.112939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d242d-8048-426f-851f-00d111033d3a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.127595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-012d917e-1b82-47ce-82ac-bfe364802ede None None] MFA Rules not processed for user `1bb6b95759d642fe9dbc24793089a70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:44.128869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-012d917e-1b82-47ce-82ac-bfe364802ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:44.162960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:44.174277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:44.174521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c7ad0-e79a-4f92-a12d-0a7d4c9d9655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.175191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c7ad0-e79a-4f92-a12d-0a7d4c9d9655 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.176006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c7ad0-e79a-4f92-a12d-0a7d4c9d9655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.176006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c7ad0-e79a-4f92-a12d-0a7d4c9d9655 admin admin] PATH_INFO: `/v3/projects/375ce88986dc47b08134e076562eee05/users/8755b16a18734175b691f891351d5116/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.192447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c7ad0-e79a-4f92-a12d-0a7d4c9d9655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375ce88986dc47b08134e076562eee05, user_id=8755b16a18734175b691f891351d5116, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.195277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c7ad0-e79a-4f92-a12d-0a7d4c9d9655 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.209284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 803/1582] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:44 2022] PUT /identity/v3/projects/375ce88986dc47b08134e076562eee05/users/8755b16a18734175b691f891351d5116/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:44.210211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c7ad0-e79a-4f92-a12d-0a7d4c9d9655 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.215547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 780/1583] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:44.216423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d917e-1b82-47ce-82ac-bfe364802ede None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.226544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d917e-1b82-47ce-82ac-bfe364802ede None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.275809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c7ad0-e79a-4f92-a12d-0a7d4c9d9655 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.280475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccee70-c1ff-4fc4-95cb-95cdfa1f08dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.281064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccee70-c1ff-4fc4-95cb-95cdfa1f08dd admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.281361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccee70-c1ff-4fc4-95cb-95cdfa1f08dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.281630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccee70-c1ff-4fc4-95cb-95cdfa1f08dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.282245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccee70-c1ff-4fc4-95cb-95cdfa1f08dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.283160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cf347-7a4e-4fd2-b1c1-a1f607a5b3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.283766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cf347-7a4e-4fd2-b1c1-a1f607a5b3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.284038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cf347-7a4e-4fd2-b1c1-a1f607a5b3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.284152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccee70-c1ff-4fc4-95cb-95cdfa1f08dd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.284306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cf347-7a4e-4fd2-b1c1-a1f607a5b3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.284980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cf347-7a4e-4fd2-b1c1-a1f607a5b3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.286538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cf347-7a4e-4fd2-b1c1-a1f607a5b3bd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.297028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 781/1584] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.297830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cf347-7a4e-4fd2-b1c1-a1f607a5b3bd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.308270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cf347-7a4e-4fd2-b1c1-a1f607a5b3bd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.312918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b16294-bf84-4115-90d0-5a0ea5cdc1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.313499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b16294-bf84-4115-90d0-5a0ea5cdc1ad admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.313774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b16294-bf84-4115-90d0-5a0ea5cdc1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.314041 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b16294-bf84-4115-90d0-5a0ea5cdc1ad admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.317346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 804/1585] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:44.317484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b16294-bf84-4115-90d0-5a0ea5cdc1ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.317596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccee70-c1ff-4fc4-95cb-95cdfa1f08dd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.317689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b16294-bf84-4115-90d0-5a0ea5cdc1ad admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.343158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 782/1586] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:44.343158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b16294-bf84-4115-90d0-5a0ea5cdc1ad admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.350505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b16294-bf84-4115-90d0-5a0ea5cdc1ad admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.379482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccee70-c1ff-4fc4-95cb-95cdfa1f08dd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.385018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514fc0f-c9f6-4396-b007-7409b7413db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.385640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514fc0f-c9f6-4396-b007-7409b7413db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.385919 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514fc0f-c9f6-4396-b007-7409b7413db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.386189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514fc0f-c9f6-4396-b007-7409b7413db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.386834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514fc0f-c9f6-4396-b007-7409b7413db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.388481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514fc0f-c9f6-4396-b007-7409b7413db1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.400256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 805/1587] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.401222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514fc0f-c9f6-4396-b007-7409b7413db1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.409322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8dc5a-c100-41e3-a94f-67dd97fe354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.409940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8dc5a-c100-41e3-a94f-67dd97fe354e admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.410217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8dc5a-c100-41e3-a94f-67dd97fe354e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.410512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8dc5a-c100-41e3-a94f-67dd97fe354e admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.411067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514fc0f-c9f6-4396-b007-7409b7413db1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.411200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8dc5a-c100-41e3-a94f-67dd97fe354e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.413232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8dc5a-c100-41e3-a94f-67dd97fe354e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.415843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ce061-62c5-4dfa-82d1-0040594dd14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.416467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ce061-62c5-4dfa-82d1-0040594dd14f admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.416757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ce061-62c5-4dfa-82d1-0040594dd14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.417051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ce061-62c5-4dfa-82d1-0040594dd14f admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.417678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ce061-62c5-4dfa-82d1-0040594dd14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.419247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ce061-62c5-4dfa-82d1-0040594dd14f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.430283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 806/1588] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.431479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54286a-c527-48ea-b7d8-137c6d74a553 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.431753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54286a-c527-48ea-b7d8-137c6d74a553 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.432018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54286a-c527-48ea-b7d8-137c6d74a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.441805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 783/1589] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:44.441805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8dc5a-c100-41e3-a94f-67dd97fe354e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.449753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8dc5a-c100-41e3-a94f-67dd97fe354e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.450815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7b54286a-c527-48ea-b7d8-137c6d74a553 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:44.451994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7b54286a-c527-48ea-b7d8-137c6d74a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:44.454575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be3c18-d3f0-4d25-8036-80929bb9b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.455216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be3c18-d3f0-4d25-8036-80929bb9b276 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.455490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be3c18-d3f0-4d25-8036-80929bb9b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.455761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be3c18-d3f0-4d25-8036-80929bb9b276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.456375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be3c18-d3f0-4d25-8036-80929bb9b276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.458279 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be3c18-d3f0-4d25-8036-80929bb9b276 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.459894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 807/1590] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:44.460702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54286a-c527-48ea-b7d8-137c6d74a553 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.471659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54286a-c527-48ea-b7d8-137c6d74a553 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.476368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b1e91-0824-41ad-9e7f-66bb7b471def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.477556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b1e91-0824-41ad-9e7f-66bb7b471def admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.478000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b1e91-0824-41ad-9e7f-66bb7b471def admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.478727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b1e91-0824-41ad-9e7f-66bb7b471def admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.478856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b1e91-0824-41ad-9e7f-66bb7b471def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.480617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b1e91-0824-41ad-9e7f-66bb7b471def admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.494232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 808/1591] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.495117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b1e91-0824-41ad-9e7f-66bb7b471def admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.507725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 784/1592] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:44.507725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be3c18-d3f0-4d25-8036-80929bb9b276 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.508083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b1e91-0824-41ad-9e7f-66bb7b471def admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.526547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:44.535211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:44.571775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af57f39-d879-4b55-8718-c9c2e767ced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.572383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af57f39-d879-4b55-8718-c9c2e767ced9 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.572688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af57f39-d879-4b55-8718-c9c2e767ced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.572961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af57f39-d879-4b55-8718-c9c2e767ced9 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.573184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be3c18-d3f0-4d25-8036-80929bb9b276 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.573613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af57f39-d879-4b55-8718-c9c2e767ced9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.575542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af57f39-d879-4b55-8718-c9c2e767ced9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.577942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e7d39-11be-44e0-a977-81dc397171fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.578547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e7d39-11be-44e0-a977-81dc397171fa admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.578823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e7d39-11be-44e0-a977-81dc397171fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.579093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e7d39-11be-44e0-a977-81dc397171fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.579732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e7d39-11be-44e0-a977-81dc397171fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.581332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e7d39-11be-44e0-a977-81dc397171fa admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.592622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 785/1593] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.593555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e7d39-11be-44e0-a977-81dc397171fa admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.599317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 809/1594] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:44.600109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af57f39-d879-4b55-8718-c9c2e767ced9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.603746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e7d39-11be-44e0-a977-81dc397171fa admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.608708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0896a-f2d8-494c-90e8-91a4d4e71217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.609342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0896a-f2d8-494c-90e8-91a4d4e71217 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.609616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0896a-f2d8-494c-90e8-91a4d4e71217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.609873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af57f39-d879-4b55-8718-c9c2e767ced9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.610001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0896a-f2d8-494c-90e8-91a4d4e71217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.610519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0896a-f2d8-494c-90e8-91a4d4e71217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.612088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0896a-f2d8-494c-90e8-91a4d4e71217 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.615000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8842-12ae-4ddd-a807-d6de8362e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.615639 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8842-12ae-4ddd-a807-d6de8362e56d admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.615916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8842-12ae-4ddd-a807-d6de8362e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.616189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8842-12ae-4ddd-a807-d6de8362e56d admin admin] PATH_INFO: `/v3/projects/8c20c9a46a9244a6b51e4bae06052d85/users/caf856d042ce4df5a0779c78e4da0658/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.623495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 786/1595] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.624137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0896a-f2d8-494c-90e8-91a4d4e71217 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.632023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e8842-12ae-4ddd-a807-d6de8362e56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c20c9a46a9244a6b51e4bae06052d85, user_id=caf856d042ce4df5a0779c78e4da0658, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.634532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0896a-f2d8-494c-90e8-91a4d4e71217 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.635330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e8842-12ae-4ddd-a807-d6de8362e56d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.639226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280cf8a-3f86-42fa-9d1b-037b9055dda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.639822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280cf8a-3f86-42fa-9d1b-037b9055dda8 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.640103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280cf8a-3f86-42fa-9d1b-037b9055dda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.640381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280cf8a-3f86-42fa-9d1b-037b9055dda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.641048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280cf8a-3f86-42fa-9d1b-037b9055dda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.642642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280cf8a-3f86-42fa-9d1b-037b9055dda8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.648223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 810/1596] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:44 2022] PUT /identity/v3/projects/8c20c9a46a9244a6b51e4bae06052d85/users/caf856d042ce4df5a0779c78e4da0658/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:44.649311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8842-12ae-4ddd-a807-d6de8362e56d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.656344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 787/1597] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.656344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280cf8a-3f86-42fa-9d1b-037b9055dda8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.663737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280cf8a-3f86-42fa-9d1b-037b9055dda8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.715086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8842-12ae-4ddd-a807-d6de8362e56d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.718702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c807-af35-49ac-bc01-e8ddb0c7533d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.719303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c807-af35-49ac-bc01-e8ddb0c7533d service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.719721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c807-af35-49ac-bc01-e8ddb0c7533d service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.720001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c807-af35-49ac-bc01-e8ddb0c7533d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.722797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3217266-cd49-4d46-8ebb-69440e229e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.723717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3217266-cd49-4d46-8ebb-69440e229e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.724001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3217266-cd49-4d46-8ebb-69440e229e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.724277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3217266-cd49-4d46-8ebb-69440e229e98 admin admin] PATH_INFO: `/v3/projects/d2ce0ec52d0245b99b49738dc8f34f73/users/f9b152fcdf9c4427830eeaba76870aa1/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.729722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030c807-af35-49ac-bc01-e8ddb0c7533d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.731162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030c807-af35-49ac-bc01-e8ddb0c7533d service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.740111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3217266-cd49-4d46-8ebb-69440e229e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ce0ec52d0245b99b49738dc8f34f73, user_id=f9b152fcdf9c4427830eeaba76870aa1, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.740629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 811/1598] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:44.741232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c807-af35-49ac-bc01-e8ddb0c7533d service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.743629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3217266-cd49-4d46-8ebb-69440e229e98 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.751769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c807-af35-49ac-bc01-e8ddb0c7533d service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.757949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea30de-b5a9-4f8e-a23d-f6b3de34f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.759006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea30de-b5a9-4f8e-a23d-f6b3de34f655 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.759408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea30de-b5a9-4f8e-a23d-f6b3de34f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.759766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea30de-b5a9-4f8e-a23d-f6b3de34f655 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.760312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea30de-b5a9-4f8e-a23d-f6b3de34f655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.762663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea30de-b5a9-4f8e-a23d-f6b3de34f655 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.764553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 788/1599] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:44 2022] PUT /identity/v3/projects/d2ce0ec52d0245b99b49738dc8f34f73/users/f9b152fcdf9c4427830eeaba76870aa1/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:44.765892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3217266-cd49-4d46-8ebb-69440e229e98 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.787226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 812/1600] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:44.788060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea30de-b5a9-4f8e-a23d-f6b3de34f655 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.801444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea30de-b5a9-4f8e-a23d-f6b3de34f655 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.864651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3217266-cd49-4d46-8ebb-69440e229e98 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.872347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bcd0e-a870-4bce-b822-ab8fb512c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.873640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bcd0e-a870-4bce-b822-ab8fb512c02e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.873640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bcd0e-a870-4bce-b822-ab8fb512c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.873867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bcd0e-a870-4bce-b822-ab8fb512c02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.874474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bcd0e-a870-4bce-b822-ab8fb512c02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.876073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bcd0e-a870-4bce-b822-ab8fb512c02e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.882432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4857094-0c06-43ff-bf9a-5a5b34b93e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.883045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4857094-0c06-43ff-bf9a-5a5b34b93e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.884301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4857094-0c06-43ff-bf9a-5a5b34b93e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.884602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4857094-0c06-43ff-bf9a-5a5b34b93e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.885249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4857094-0c06-43ff-bf9a-5a5b34b93e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.886849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4857094-0c06-43ff-bf9a-5a5b34b93e03 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.891255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 789/1601] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.892183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bcd0e-a870-4bce-b822-ab8fb512c02e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.901124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 813/1602] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.901124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4857094-0c06-43ff-bf9a-5a5b34b93e03 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:44.904714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bcd0e-a870-4bce-b822-ab8fb512c02e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.911605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557c22e-1c90-4ef1-a356-10a6efc3fc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.912271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557c22e-1c90-4ef1-a356-10a6efc3fc5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.912574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557c22e-1c90-4ef1-a356-10a6efc3fc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.912840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557c22e-1c90-4ef1-a356-10a6efc3fc5b admin admin] PATH_INFO: `/v3/projects/df78df5c522b42b1b40aebf2502866e7/users/5464533ccfb74094bf613bcd51c55ff1/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.913629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4857094-0c06-43ff-bf9a-5a5b34b93e03 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:44.920536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bd3bd-b0d7-40d0-83fe-17dd75cc4e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:44.921200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd3bd-b0d7-40d0-83fe-17dd75cc4e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.921614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd3bd-b0d7-40d0-83fe-17dd75cc4e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.921897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd3bd-b0d7-40d0-83fe-17dd75cc4e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.923142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bd3bd-b0d7-40d0-83fe-17dd75cc4e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.924065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:44.926314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bd3bd-b0d7-40d0-83fe-17dd75cc4e43 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.933937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557c22e-1c90-4ef1-a356-10a6efc3fc5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df78df5c522b42b1b40aebf2502866e7, user_id=5464533ccfb74094bf613bcd51c55ff1, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:44.936872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557c22e-1c90-4ef1-a356-10a6efc3fc5b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:44.942916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 814/1603] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:44.944248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5919bf-dbcf-4938-83ac-3aae8595b48c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.944584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5919bf-dbcf-4938-83ac-3aae8595b48c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.944866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5919bf-dbcf-4938-83ac-3aae8595b48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.955804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:44.964082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 790/1604] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:44 2022] PUT /identity/v3/projects/df78df5c522b42b1b40aebf2502866e7/users/5464533ccfb74094bf613bcd51c55ff1/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:44.966952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a1f34-48cd-4838-9752-f219b0de926c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:44.967341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a1f34-48cd-4838-9752-f219b0de926c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:44.967501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a1f34-48cd-4838-9752-f219b0de926c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:44.985672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-af5919bf-dbcf-4938-83ac-3aae8595b48c None None] MFA Rules not processed for user `caf856d042ce4df5a0779c78e4da0658`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:44.987042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-af5919bf-dbcf-4938-83ac-3aae8595b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:45.000938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d84a1f34-48cd-4838-9752-f219b0de926c None None] MFA Rules not processed for user `f9b152fcdf9c4427830eeaba76870aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:45.002268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d84a1f34-48cd-4838-9752-f219b0de926c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:45.074745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 815/1605] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:45.076279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5919bf-dbcf-4938-83ac-3aae8595b48c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.095444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 791/1606] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:45.096619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a1f34-48cd-4838-9752-f219b0de926c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.162682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5919bf-dbcf-4938-83ac-3aae8595b48c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.168197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d35d-8826-40c7-9b2b-ccf960044efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:45.168919 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d35d-8826-40c7-9b2b-ccf960044efc admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.169180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d35d-8826-40c7-9b2b-ccf960044efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.169452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d35d-8826-40c7-9b2b-ccf960044efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.171844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5d35d-8826-40c7-9b2b-ccf960044efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:45.174147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5d35d-8826-40c7-9b2b-ccf960044efc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:45.186145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a1f34-48cd-4838-9752-f219b0de926c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.193008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe835f68-2cd5-417c-9259-450d1409b021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:45.194177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835f68-2cd5-417c-9259-450d1409b021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.194177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835f68-2cd5-417c-9259-450d1409b021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.194602 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835f68-2cd5-417c-9259-450d1409b021 admin admin] PATH_INFO: `/v3/projects/f37b888f1a584296a90d6dec62dbb664/users/e7f6a0023e4d49caa61ae218bf803ad9/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.200546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 816/1607] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:45.202024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d35d-8826-40c7-9b2b-ccf960044efc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.219283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d35d-8826-40c7-9b2b-ccf960044efc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.221804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe835f68-2cd5-417c-9259-450d1409b021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f37b888f1a584296a90d6dec62dbb664, user_id=e7f6a0023e4d49caa61ae218bf803ad9, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:45.229264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe835f68-2cd5-417c-9259-450d1409b021 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:45.230452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b2456-ac52-4f84-a847-7da46efc4ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:45.231588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b2456-ac52-4f84-a847-7da46efc4ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.232207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b2456-ac52-4f84-a847-7da46efc4ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.232808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b2456-ac52-4f84-a847-7da46efc4ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.234011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b2456-ac52-4f84-a847-7da46efc4ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:45.237457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b2456-ac52-4f84-a847-7da46efc4ec1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:45.255582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 817/1608] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:45.260260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b2456-ac52-4f84-a847-7da46efc4ec1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.270059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 792/1609] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:45 2022] PUT /identity/v3/projects/f37b888f1a584296a90d6dec62dbb664/users/e7f6a0023e4d49caa61ae218bf803ad9/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:45.271655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ea9a0-7ade-45f8-a0e8-a175ebdb0444 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.271931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ea9a0-7ade-45f8-a0e8-a175ebdb0444 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.272410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ea9a0-7ade-45f8-a0e8-a175ebdb0444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.289966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b2456-ac52-4f84-a847-7da46efc4ec1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.305761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0b8ea9a0-7ade-45f8-a0e8-a175ebdb0444 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:45.307460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0b8ea9a0-7ade-45f8-a0e8-a175ebdb0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:45.366404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e46378-6832-4718-8c43-66a8295eb193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:45.367076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e46378-6832-4718-8c43-66a8295eb193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.367362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e46378-6832-4718-8c43-66a8295eb193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.368208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e46378-6832-4718-8c43-66a8295eb193 admin admin] PATH_INFO: `/v3/projects/8b32d5c45c614fd08a57af6be1143590/users/e9ad80a549b14b25ac71cd96832a74e7/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.384605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:45.399974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 793/1610] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:45.401552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ea9a0-7ade-45f8-a0e8-a175ebdb0444 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.402466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e46378-6832-4718-8c43-66a8295eb193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b32d5c45c614fd08a57af6be1143590, user_id=e9ad80a549b14b25ac71cd96832a74e7, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:45.407255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e46378-6832-4718-8c43-66a8295eb193 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:45.427478 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:45.431763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 818/1611] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:45 2022] PUT /identity/v3/projects/8b32d5c45c614fd08a57af6be1143590/users/e9ad80a549b14b25ac71cd96832a74e7/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:45.432964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e46378-6832-4718-8c43-66a8295eb193 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.476925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ea9a0-7ade-45f8-a0e8-a175ebdb0444 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.530740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e46378-6832-4718-8c43-66a8295eb193 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.536769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859b96e-e762-4bfb-a6d1-f35e1e170efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:45.537816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859b96e-e762-4bfb-a6d1-f35e1e170efd admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.538718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859b96e-e762-4bfb-a6d1-f35e1e170efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.538718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859b96e-e762-4bfb-a6d1-f35e1e170efd admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.539350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859b96e-e762-4bfb-a6d1-f35e1e170efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:45.542112 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859b96e-e762-4bfb-a6d1-f35e1e170efd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:45.558012 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ec167-c1b1-45b8-b3a3-aec2517972c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:45.558638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ec167-c1b1-45b8-b3a3-aec2517972c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.558916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ec167-c1b1-45b8-b3a3-aec2517972c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.559339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ec167-c1b1-45b8-b3a3-aec2517972c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.582914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 819/1612] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:45 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:45.583605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859b96e-e762-4bfb-a6d1-f35e1e170efd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.630997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ec167-c1b1-45b8-b3a3-aec2517972c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:45.632542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ec167-c1b1-45b8-b3a3-aec2517972c5 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:45.643832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 794/1613] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:45.644704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ec167-c1b1-45b8-b3a3-aec2517972c5 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.657435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ec167-c1b1-45b8-b3a3-aec2517972c5 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.663057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859b96e-e762-4bfb-a6d1-f35e1e170efd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.665859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c64ae6-2fe8-4d8f-9d43-69c7c2f6068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:45.667061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c64ae6-2fe8-4d8f-9d43-69c7c2f6068d admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.667661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c64ae6-2fe8-4d8f-9d43-69c7c2f6068d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.668134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c64ae6-2fe8-4d8f-9d43-69c7c2f6068d admin admin] PATH_INFO: `/v3/projects/e650ca5f46214650b89cfba0e33bd957/users/be3f93f1745a47549e6b6edc544278c6/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.669040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e669c-8722-426c-af40-6e471bbbbb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:45.671971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e669c-8722-426c-af40-6e471bbbbb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.671971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e669c-8722-426c-af40-6e471bbbbb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.671971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e669c-8722-426c-af40-6e471bbbbb28 admin admin] PATH_INFO: `/v3/projects/4788c8cb0e304226a8ab2b38cec17219/users/2534915a0ec9465d8abb4aea9a644958/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.693824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c64ae6-2fe8-4d8f-9d43-69c7c2f6068d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e650ca5f46214650b89cfba0e33bd957, user_id=be3f93f1745a47549e6b6edc544278c6, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:45.694353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e669c-8722-426c-af40-6e471bbbbb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4788c8cb0e304226a8ab2b38cec17219, user_id=2534915a0ec9465d8abb4aea9a644958, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:45.697242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c64ae6-2fe8-4d8f-9d43-69c7c2f6068d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:45.697796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e669c-8722-426c-af40-6e471bbbbb28 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:45.717149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 820/1614] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:45 2022] PUT /identity/v3/projects/4788c8cb0e304226a8ab2b38cec17219/users/2534915a0ec9465d8abb4aea9a644958/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:45.722529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9713-0d83-4770-9af5-6d4aae2aa0ba None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.722761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 795/1615] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:45 2022] PUT /identity/v3/projects/e650ca5f46214650b89cfba0e33bd957/users/be3f93f1745a47549e6b6edc544278c6/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:45.723321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9713-0d83-4770-9af5-6d4aae2aa0ba None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.723846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9713-0d83-4770-9af5-6d4aae2aa0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.725366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c64ae6-2fe8-4d8f-9d43-69c7c2f6068d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:45.778356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-52be9713-0d83-4770-9af5-6d4aae2aa0ba None None] MFA Rules not processed for user `e7f6a0023e4d49caa61ae218bf803ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:45.781217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-52be9713-0d83-4770-9af5-6d4aae2aa0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:45.866171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:45.872787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:45.879718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c64ae6-2fe8-4d8f-9d43-69c7c2f6068d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:45.889251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 821/1616] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:45.890375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ee947-62ad-409f-87f8-63d793af7666 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:45.890880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ee947-62ad-409f-87f8-63d793af7666 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:45.891277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ee947-62ad-409f-87f8-63d793af7666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:45.934316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a07ee947-62ad-409f-87f8-63d793af7666 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:45.935560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a07ee947-62ad-409f-87f8-63d793af7666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:46.002679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5d159-3512-4f01-a310-807a90425f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.004523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5d159-3512-4f01-a310-807a90425f70 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.005133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5d159-3512-4f01-a310-807a90425f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.007541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5d159-3512-4f01-a310-807a90425f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.065103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 822/1617] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:46.065103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ee947-62ad-409f-87f8-63d793af7666 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.091546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ee947-62ad-409f-87f8-63d793af7666 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.099839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9593ea-053a-4468-aa67-e0751ff4cf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.100544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9593ea-053a-4468-aa67-e0751ff4cf79 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.101080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9593ea-053a-4468-aa67-e0751ff4cf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.101204 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9593ea-053a-4468-aa67-e0751ff4cf79 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.102207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9593ea-053a-4468-aa67-e0751ff4cf79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.110068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9593ea-053a-4468-aa67-e0751ff4cf79 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.130900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5d159-3512-4f01-a310-807a90425f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.135505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5d159-3512-4f01-a310-807a90425f70 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.156472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 823/1618] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:46 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:46.159405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9593ea-053a-4468-aa67-e0751ff4cf79 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.168669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 796/1619] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:46.172566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4f52f-8982-46ea-b138-463617c896e4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.173125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4f52f-8982-46ea-b138-463617c896e4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.174867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4f52f-8982-46ea-b138-463617c896e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.198713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9593ea-053a-4468-aa67-e0751ff4cf79 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.216268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9fd4f52f-8982-46ea-b138-463617c896e4 None None] MFA Rules not processed for user `be3f93f1745a47549e6b6edc544278c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:46.218876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9fd4f52f-8982-46ea-b138-463617c896e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:46.223921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528bad-80b8-4c2c-adac-fcd303ef0479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.224657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528bad-80b8-4c2c-adac-fcd303ef0479 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.224938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528bad-80b8-4c2c-adac-fcd303ef0479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.225217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528bad-80b8-4c2c-adac-fcd303ef0479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.228928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9528bad-80b8-4c2c-adac-fcd303ef0479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.232027 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9528bad-80b8-4c2c-adac-fcd303ef0479 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.256024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 824/1620] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:46.259212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528bad-80b8-4c2c-adac-fcd303ef0479 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.283699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528bad-80b8-4c2c-adac-fcd303ef0479 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.288831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c12b7-f866-4ca0-87d6-8afee03a22aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.289995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c12b7-f866-4ca0-87d6-8afee03a22aa admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.290293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c12b7-f866-4ca0-87d6-8afee03a22aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.290566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c12b7-f866-4ca0-87d6-8afee03a22aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.291198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c12b7-f866-4ca0-87d6-8afee03a22aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.293133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c12b7-f866-4ca0-87d6-8afee03a22aa admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.306061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 797/1621] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:46 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:46.306835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4f52f-8982-46ea-b138-463617c896e4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.326503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4f52f-8982-46ea-b138-463617c896e4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.337078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91572f77-1aea-4506-8119-7bb626c7e43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.337872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572f77-1aea-4506-8119-7bb626c7e43b service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.338246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572f77-1aea-4506-8119-7bb626c7e43b service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.338783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572f77-1aea-4506-8119-7bb626c7e43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.351182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91572f77-1aea-4506-8119-7bb626c7e43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.354010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91572f77-1aea-4506-8119-7bb626c7e43b service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.366265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 798/1622] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:46.367993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91572f77-1aea-4506-8119-7bb626c7e43b service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.369264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 825/1623] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:46 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:46.370721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c12b7-f866-4ca0-87d6-8afee03a22aa admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.385292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91572f77-1aea-4506-8119-7bb626c7e43b service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.417245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:46.464468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e188cf-ce95-4710-be7c-e16c7ef40e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.466063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e188cf-ce95-4710-be7c-e16c7ef40e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.466446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e188cf-ce95-4710-be7c-e16c7ef40e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.467689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e188cf-ce95-4710-be7c-e16c7ef40e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.469589 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e188cf-ce95-4710-be7c-e16c7ef40e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.472272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e188cf-ce95-4710-be7c-e16c7ef40e8e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.499054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 799/1624] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:46.501099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e188cf-ce95-4710-be7c-e16c7ef40e8e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.527764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c12b7-f866-4ca0-87d6-8afee03a22aa admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.536299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece66161-eff9-45d5-a6d3-02e2b4dc1a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.537203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece66161-eff9-45d5-a6d3-02e2b4dc1a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.537638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece66161-eff9-45d5-a6d3-02e2b4dc1a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.538022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece66161-eff9-45d5-a6d3-02e2b4dc1a20 admin admin] PATH_INFO: `/v3/projects/4788c8cb0e304226a8ab2b38cec17219/users/2534915a0ec9465d8abb4aea9a644958/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.539054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e188cf-ce95-4710-be7c-e16c7ef40e8e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.545306 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece66161-eff9-45d5-a6d3-02e2b4dc1a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4788c8cb0e304226a8ab2b38cec17219, user_id=2534915a0ec9465d8abb4aea9a644958, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.553487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c1235-8372-4a79-a1df-105421c5a92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.556787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1235-8372-4a79-a1df-105421c5a92a admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.557079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1235-8372-4a79-a1df-105421c5a92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.557498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1235-8372-4a79-a1df-105421c5a92a admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.558400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c1235-8372-4a79-a1df-105421c5a92a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.558748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece66161-eff9-45d5-a6d3-02e2b4dc1a20 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.560897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c1235-8372-4a79-a1df-105421c5a92a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.589826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 826/1625] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:46 2022] PUT /identity/v3/projects/4788c8cb0e304226a8ab2b38cec17219/users/2534915a0ec9465d8abb4aea9a644958/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:46.592506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece66161-eff9-45d5-a6d3-02e2b4dc1a20 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.661902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:46.678380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 800/1626] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:46 2022] POST /identity/v3/projects => generated 405 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:46.680800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c1235-8372-4a79-a1df-105421c5a92a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:46.705485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece66161-eff9-45d5-a6d3-02e2b4dc1a20 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.865831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c1235-8372-4a79-a1df-105421c5a92a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:46.876971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed27b73-b7ff-42fb-ace6-3f4b05306655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.877676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed27b73-b7ff-42fb-ace6-3f4b05306655 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.877956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed27b73-b7ff-42fb-ace6-3f4b05306655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.878207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed27b73-b7ff-42fb-ace6-3f4b05306655 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.879132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed27b73-b7ff-42fb-ace6-3f4b05306655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.881126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed27b73-b7ff-42fb-ace6-3f4b05306655 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:46.881485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59db37e-e29b-4818-baca-07b941b482c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:46.922220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59db37e-e29b-4818-baca-07b941b482c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.923417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59db37e-e29b-4818-baca-07b941b482c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.923714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59db37e-e29b-4818-baca-07b941b482c1 admin admin] PATH_INFO: `/v3/projects/de3cf5b2a95a430c944d315eabf00d9f/users/3d92be02836d4b889e531f646b7b5b6e/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.932628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 827/1627] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:46 2022] POST /identity/v3/users => generated 300 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:46.934006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756113af-8350-4a67-b2ff-7cf41c354a42 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:46.934284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756113af-8350-4a67-b2ff-7cf41c354a42 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:46.934553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756113af-8350-4a67-b2ff-7cf41c354a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:46.966754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59db37e-e29b-4818-baca-07b941b482c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3cf5b2a95a430c944d315eabf00d9f, user_id=3d92be02836d4b889e531f646b7b5b6e, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:46.971694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59db37e-e29b-4818-baca-07b941b482c1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.010640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 801/1628] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:46 2022] PUT /identity/v3/projects/de3cf5b2a95a430c944d315eabf00d9f/users/3d92be02836d4b889e531f646b7b5b6e/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:47.012718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59db37e-e29b-4818-baca-07b941b482c1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.022456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-756113af-8350-4a67-b2ff-7cf41c354a42 None None] MFA Rules not processed for user `2534915a0ec9465d8abb4aea9a644958`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:47.026980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-756113af-8350-4a67-b2ff-7cf41c354a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:47.304755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59db37e-e29b-4818-baca-07b941b482c1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:47.333854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15d6bd-8052-444b-b241-e0e8319e3a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:47.334869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d6bd-8052-444b-b241-e0e8319e3a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:47.335313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d6bd-8052-444b-b241-e0e8319e3a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:47.335890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d6bd-8052-444b-b241-e0e8319e3a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:47.337373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15d6bd-8052-444b-b241-e0e8319e3a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:47.350831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15d6bd-8052-444b-b241-e0e8319e3a5f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.377382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 828/1629] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:47.397221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756113af-8350-4a67-b2ff-7cf41c354a42 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.428060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 802/1630] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:47 2022] POST /identity/v3/users => generated 309 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:47.434036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15d6bd-8052-444b-b241-e0e8319e3a5f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.442710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756113af-8350-4a67-b2ff-7cf41c354a42 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:47.455165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc916700-1286-4b87-b608-00f651be30ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:47.457669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916700-1286-4b87-b608-00f651be30ee admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:47.458309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916700-1286-4b87-b608-00f651be30ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:47.458851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916700-1286-4b87-b608-00f651be30ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:47.460240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc916700-1286-4b87-b608-00f651be30ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:47.465811 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc916700-1286-4b87-b608-00f651be30ee admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.471162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15d6bd-8052-444b-b241-e0e8319e3a5f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:47.496475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 829/1631] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:47.496475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc916700-1286-4b87-b608-00f651be30ee admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.498743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90277116-23c6-4259-9745-b319fc95f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:47.500095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90277116-23c6-4259-9745-b319fc95f353 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:47.500940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90277116-23c6-4259-9745-b319fc95f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:47.501652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90277116-23c6-4259-9745-b319fc95f353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:47.502916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90277116-23c6-4259-9745-b319fc95f353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:47.511287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90277116-23c6-4259-9745-b319fc95f353 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.544256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 803/1632] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:47.544256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90277116-23c6-4259-9745-b319fc95f353 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.553951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:47.564881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90277116-23c6-4259-9745-b319fc95f353 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:47.576017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbc0b2-b927-460e-996f-bec5d869e3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:47.577031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbc0b2-b927-460e-996f-bec5d869e3ae admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:47.577499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbc0b2-b927-460e-996f-bec5d869e3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:47.578177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbc0b2-b927-460e-996f-bec5d869e3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:47.579240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbc0b2-b927-460e-996f-bec5d869e3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:47.582252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbc0b2-b927-460e-996f-bec5d869e3ae admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.606441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 804/1633] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:47.606441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbc0b2-b927-460e-996f-bec5d869e3ae admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.622813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbc0b2-b927-460e-996f-bec5d869e3ae admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:47.626898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc916700-1286-4b87-b608-00f651be30ee admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:47.627997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcb972-44ca-4cd6-bb3e-2a16a7f56e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:47.628913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcb972-44ca-4cd6-bb3e-2a16a7f56e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:47.629205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcb972-44ca-4cd6-bb3e-2a16a7f56e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:47.629498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcb972-44ca-4cd6-bb3e-2a16a7f56e72 admin admin] PATH_INFO: `/v3/projects/984d23ece00f4aa8bf257331db874024/users/7f85b5b3e27642fda3fabac620e88b8e/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:47.631319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ec5cd-92af-405c-85e1-475095ec4ac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:47.631656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ec5cd-92af-405c-85e1-475095ec4ac1 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:47.632064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ec5cd-92af-405c-85e1-475095ec4ac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:47.632415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ec5cd-92af-405c-85e1-475095ec4ac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:47.641554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:47.644057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ec5cd-92af-405c-85e1-475095ec4ac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:47.645816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ec5cd-92af-405c-85e1-475095ec4ac1 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.651246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcb972-44ca-4cd6-bb3e-2a16a7f56e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984d23ece00f4aa8bf257331db874024, user_id=7f85b5b3e27642fda3fabac620e88b8e, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:47.663148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcb972-44ca-4cd6-bb3e-2a16a7f56e72 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.663275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 830/1634] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:47.667183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ec5cd-92af-405c-85e1-475095ec4ac1 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.699357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ec5cd-92af-405c-85e1-475095ec4ac1 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:47.710799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 805/1635] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:47 2022] PUT /identity/v3/projects/984d23ece00f4aa8bf257331db874024/users/7f85b5b3e27642fda3fabac620e88b8e/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:47.713576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcb972-44ca-4cd6-bb3e-2a16a7f56e72 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.863754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcb972-44ca-4cd6-bb3e-2a16a7f56e72 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:47.865851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b9e72-8715-491b-a401-046964702d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:47.868480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b9e72-8715-491b-a401-046964702d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:47.868953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b9e72-8715-491b-a401-046964702d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:47.870151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b9e72-8715-491b-a401-046964702d4e admin admin] PATH_INFO: `/v3/projects/de3cf5b2a95a430c944d315eabf00d9f/users/3d92be02836d4b889e531f646b7b5b6e/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:47.874789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b9e72-8715-491b-a401-046964702d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3cf5b2a95a430c944d315eabf00d9f, user_id=3d92be02836d4b889e531f646b7b5b6e, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:47.879781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bee4ea-83ea-4c3a-8c0a-27cdaae4923e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:47.882206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee4ea-83ea-4c3a-8c0a-27cdaae4923e admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:47.882764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee4ea-83ea-4c3a-8c0a-27cdaae4923e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:47.884724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b9e72-8715-491b-a401-046964702d4e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.885292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee4ea-83ea-4c3a-8c0a-27cdaae4923e admin admin] PATH_INFO: `/v3/projects/425c8090bcf3423ca3a9984ff8c61a60/users/b9a33353b1db4adc816dd83fe140fb7a/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:47.928428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 831/1636] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:47 2022] PUT /identity/v3/projects/de3cf5b2a95a430c944d315eabf00d9f/users/3d92be02836d4b889e531f646b7b5b6e/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:47.928428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b9e72-8715-491b-a401-046964702d4e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:47.937396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bee4ea-83ea-4c3a-8c0a-27cdaae4923e admin admin] RBAC: Authorizing `identity:create_grant(project_id=425c8090bcf3423ca3a9984ff8c61a60, user_id=b9a33353b1db4adc816dd83fe140fb7a, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:47.943019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bee4ea-83ea-4c3a-8c0a-27cdaae4923e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:47.979969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 806/1637] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:47 2022] PUT /identity/v3/projects/425c8090bcf3423ca3a9984ff8c61a60/users/b9a33353b1db4adc816dd83fe140fb7a/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:47.981162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bee4ea-83ea-4c3a-8c0a-27cdaae4923e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:48.103043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b9e72-8715-491b-a401-046964702d4e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:48.148405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bee4ea-83ea-4c3a-8c0a-27cdaae4923e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:48.168354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287dad5-50f7-43c9-b38e-cce874a35586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:48.169718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287dad5-50f7-43c9-b38e-cce874a35586 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.169718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287dad5-50f7-43c9-b38e-cce874a35586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.170386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287dad5-50f7-43c9-b38e-cce874a35586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.290272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8350f8-ef45-4899-8c20-c6bb57470a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:48.295518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8350f8-ef45-4899-8c20-c6bb57470a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.295876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8350f8-ef45-4899-8c20-c6bb57470a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.296125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8350f8-ef45-4899-8c20-c6bb57470a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.298037 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8350f8-ef45-4899-8c20-c6bb57470a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:48.303992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8350f8-ef45-4899-8c20-c6bb57470a67 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:48.337898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287dad5-50f7-43c9-b38e-cce874a35586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:48.339362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287dad5-50f7-43c9-b38e-cce874a35586 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:48.344103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 832/1638] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:48.348057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8e6d1-5d27-42e8-9327-08491e39f5b1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.348361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8e6d1-5d27-42e8-9327-08491e39f5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.350040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8e6d1-5d27-42e8-9327-08491e39f5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.360375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 807/1639] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:48.361267 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287dad5-50f7-43c9-b38e-cce874a35586 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:48.393945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287dad5-50f7-43c9-b38e-cce874a35586 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:48.409160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fe8ab-9bb7-4706-87ad-2ffcd0270ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:48.409789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fe8ab-9bb7-4706-87ad-2ffcd0270ddc admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.410065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fe8ab-9bb7-4706-87ad-2ffcd0270ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.410335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fe8ab-9bb7-4706-87ad-2ffcd0270ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.411154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fe8ab-9bb7-4706-87ad-2ffcd0270ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:48.412833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-64a8e6d1-5d27-42e8-9327-08491e39f5b1 None None] MFA Rules not processed for user `3d92be02836d4b889e531f646b7b5b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:48.413270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fe8ab-9bb7-4706-87ad-2ffcd0270ddc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:48.414087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-64a8e6d1-5d27-42e8-9327-08491e39f5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:48.417695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:48.440999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 808/1640] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:48.440999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fe8ab-9bb7-4706-87ad-2ffcd0270ddc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:48.466627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fe8ab-9bb7-4706-87ad-2ffcd0270ddc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:48.479334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f385a9-6bd7-42d1-ae0e-65c8032e4eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:48.479950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:48.480070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f385a9-6bd7-42d1-ae0e-65c8032e4eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.480288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f385a9-6bd7-42d1-ae0e-65c8032e4eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.480590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f385a9-6bd7-42d1-ae0e-65c8032e4eb4 admin admin] PATH_INFO: `/v3/projects/984d23ece00f4aa8bf257331db874024/users/7f85b5b3e27642fda3fabac620e88b8e/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.485141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f385a9-6bd7-42d1-ae0e-65c8032e4eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984d23ece00f4aa8bf257331db874024, user_id=7f85b5b3e27642fda3fabac620e88b8e, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:48.487822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f385a9-6bd7-42d1-ae0e-65c8032e4eb4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:48.536735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 809/1641] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:48 2022] PUT /identity/v3/projects/984d23ece00f4aa8bf257331db874024/users/7f85b5b3e27642fda3fabac620e88b8e/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:48.538152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f385a9-6bd7-42d1-ae0e-65c8032e4eb4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:48.627521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 833/1642] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:48.629049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39731-add0-471c-b609-e7eca101e2fd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.629335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39731-add0-471c-b609-e7eca101e2fd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.629606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39731-add0-471c-b609-e7eca101e2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.666105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e9f39731-add0-471c-b609-e7eca101e2fd None None] MFA Rules not processed for user `7f85b5b3e27642fda3fabac620e88b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:48.667625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e9f39731-add0-471c-b609-e7eca101e2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:48.701525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f385a9-6bd7-42d1-ae0e-65c8032e4eb4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:48.708662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8fd30-9c0f-4b20-8a16-10bce635487e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:48.710321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8fd30-9c0f-4b20-8a16-10bce635487e admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.710704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8fd30-9c0f-4b20-8a16-10bce635487e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.710996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8fd30-9c0f-4b20-8a16-10bce635487e admin admin] PATH_INFO: `/v3/projects/425c8090bcf3423ca3a9984ff8c61a60/users/b9a33353b1db4adc816dd83fe140fb7a/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.714556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8fd30-9c0f-4b20-8a16-10bce635487e admin admin] RBAC: Authorizing `identity:create_grant(project_id=425c8090bcf3423ca3a9984ff8c61a60, user_id=b9a33353b1db4adc816dd83fe140fb7a, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:48.718391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8fd30-9c0f-4b20-8a16-10bce635487e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:48.733808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 810/1643] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:48 2022] PUT /identity/v3/projects/425c8090bcf3423ca3a9984ff8c61a60/users/b9a33353b1db4adc816dd83fe140fb7a/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:48.735358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8fd30-9c0f-4b20-8a16-10bce635487e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:48.823418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 834/1644] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:48.826768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c34ed6-2957-456e-b06e-7bdb8c60ea46 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.827342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c34ed6-2957-456e-b06e-7bdb8c60ea46 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.828046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c34ed6-2957-456e-b06e-7bdb8c60ea46 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.833161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8fd30-9c0f-4b20-8a16-10bce635487e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:48.840567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 835/1645] 10.222.0.70 () {54 vars in 962 bytes} [Tue Aug 9 19:13:48 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:13:48.843001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a50f2-4107-48de-a5a5-5339d4eba421 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.843890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a50f2-4107-48de-a5a5-5339d4eba421 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.844813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a50f2-4107-48de-a5a5-5339d4eba421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.846125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ed8fa-b333-49bf-bcdf-ce36eef2f464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:48.846763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ed8fa-b333-49bf-bcdf-ce36eef2f464 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:48.847146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ed8fa-b333-49bf-bcdf-ce36eef2f464 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:48.847530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ed8fa-b333-49bf-bcdf-ce36eef2f464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:48.913025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-396a50f2-4107-48de-a5a5-5339d4eba421 None None] MFA Rules not processed for user `b9a33353b1db4adc816dd83fe140fb7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:48.917260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-396a50f2-4107-48de-a5a5-5339d4eba421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:48.998014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ed8fa-b333-49bf-bcdf-ce36eef2f464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:49.004078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ed8fa-b333-49bf-bcdf-ce36eef2f464 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:49.019084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 811/1646] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:49.031020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ed8fa-b333-49bf-bcdf-ce36eef2f464 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:49.048715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ed8fa-b333-49bf-bcdf-ce36eef2f464 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:49.062284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb1630-5952-4c8f-b77a-335d61f6efb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:49.063582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1630-5952-4c8f-b77a-335d61f6efb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:49.064501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1630-5952-4c8f-b77a-335d61f6efb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:49.064793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb1630-5952-4c8f-b77a-335d61f6efb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:49.102498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 836/1647] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:48 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:49.106634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90436-90ed-4f28-81c0-a4a5451415c9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:49.107507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90436-90ed-4f28-81c0-a4a5451415c9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:49.107718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90436-90ed-4f28-81c0-a4a5451415c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:49.122216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:49.205099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bcb90436-90ed-4f28-81c0-a4a5451415c9 None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:49.206385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bcb90436-90ed-4f28-81c0-a4a5451415c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:49.222904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:49.223731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb1630-5952-4c8f-b77a-335d61f6efb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:49.226346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb1630-5952-4c8f-b77a-335d61f6efb8 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:49.257593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 812/1648] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:49.260396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb1630-5952-4c8f-b77a-335d61f6efb8 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:49.285515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb1630-5952-4c8f-b77a-335d61f6efb8 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:49.301311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82dc55-b004-4af3-8813-0c85a05b2634 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:49.302008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82dc55-b004-4af3-8813-0c85a05b2634 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:49.302287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82dc55-b004-4af3-8813-0c85a05b2634 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:49.306454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82dc55-b004-4af3-8813-0c85a05b2634 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:49.328391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82dc55-b004-4af3-8813-0c85a05b2634 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:49.332732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82dc55-b004-4af3-8813-0c85a05b2634 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:49.350339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 837/1649] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Aug 9 19:13:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:49.351357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90436-90ed-4f28-81c0-a4a5451415c9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:49.377795 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 813/1650] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:49 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:49.378906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82dc55-b004-4af3-8813-0c85a05b2634 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:49.395842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82dc55-b004-4af3-8813-0c85a05b2634 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:49.410345 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d932de-1af7-403c-ac44-f90503317e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:49.410755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d932de-1af7-403c-ac44-f90503317e9e service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:49.411040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d932de-1af7-403c-ac44-f90503317e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:49.412466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d932de-1af7-403c-ac44-f90503317e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:49.543786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90436-90ed-4f28-81c0-a4a5451415c9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:49.558554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:49.561440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:49.561854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:49.563368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] PATH_INFO: `/v3/users/e7f6a0023e4d49caa61ae218bf803ad9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:49.565139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f6a0023e4d49caa61ae218bf803ad9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:49.565410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d932de-1af7-403c-ac44-f90503317e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:49.567668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d932de-1af7-403c-ac44-f90503317e9e service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:49.568537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:13:49.568537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Aug 09 19:13:49.568537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: Aug 09 19:13:49.571728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:49.589624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 814/1651] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:49.591932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae144b6-7755-423d-8407-ebf6a250930d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:49.592314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae144b6-7755-423d-8407-ebf6a250930d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:49.592822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae144b6-7755-423d-8407-ebf6a250930d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:49.594656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 815/1652] 10.222.0.70 () {54 vars in 912 bytes} [Tue Aug 9 19:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:13:49.612036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3c531-1505-47c6-8e41-b8036d985d51 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:49.612234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3c531-1505-47c6-8e41-b8036d985d51 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:49.612618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3c531-1505-47c6-8e41-b8036d985d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:49.695299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bbf3c531-1505-47c6-8e41-b8036d985d51 None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:49.697807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bbf3c531-1505-47c6-8e41-b8036d985d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:49.720637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 816/1653] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:13:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:49.727490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3c531-1505-47c6-8e41-b8036d985d51 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:49.749052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f6a0023e4d49caa61ae218bf803ad9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.769127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3c531-1505-47c6-8e41-b8036d985d51 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:49.772700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f6a0023e4d49caa61ae218bf803ad9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.787555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f6a0023e4d49caa61ae218bf803ad9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.788860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f6a0023e4d49caa61ae218bf803ad9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.790511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f6a0023e4d49caa61ae218bf803ad9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.791253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:49.792255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:49.792699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:49.793153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] PATH_INFO: `/v3/users/be3f93f1745a47549e6b6edc544278c6` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:49.801753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3f93f1745a47549e6b6edc544278c6)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:49.807099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:49.866618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 838/1654] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:13:49 2022] DELETE /identity/v3/users/e7f6a0023e4d49caa61ae218bf803ad9 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:49.868183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:49.873676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:49.925901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3f93f1745a47549e6b6edc544278c6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.944715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3f93f1745a47549e6b6edc544278c6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.960330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:49.963452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3f93f1745a47549e6b6edc544278c6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.964703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3f93f1745a47549e6b6edc544278c6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:49.966669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-314e9f9d-bc31-4ce7-ac22-3c2c51f5dcae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3f93f1745a47549e6b6edc544278c6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:50.034552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 817/1655] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:13:49 2022] DELETE /identity/v3/users/be3f93f1745a47549e6b6edc544278c6 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:50.035916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c16b8-6db7-4ed5-a42c-2cc37933aee0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:50.037028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c16b8-6db7-4ed5-a42c-2cc37933aee0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:50.037317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c16b8-6db7-4ed5-a42c-2cc37933aee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:50.058301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075fd9-d76e-49d7-9440-5bba506d9939 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:50.089380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1d9c16b8-6db7-4ed5-a42c-2cc37933aee0 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:50.090635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1d9c16b8-6db7-4ed5-a42c-2cc37933aee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:50.110892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:50.177820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:50.178439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:50.178719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:50.179090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:50.187446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:50.266275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 818/1656] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:50.267828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60372af-1bbe-43f6-b9b2-f7092c0de00f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:50.267828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60372af-1bbe-43f6-b9b2-f7092c0de00f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:50.268037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60372af-1bbe-43f6-b9b2-f7092c0de00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:50.322423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a60372af-1bbe-43f6-b9b2-f7092c0de00f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:50.325610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a60372af-1bbe-43f6-b9b2-f7092c0de00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:50.349098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 819/1657] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:50.352076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60372af-1bbe-43f6-b9b2-f7092c0de00f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:50.357642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a60372af-1bbe-43f6-b9b2-f7092c0de00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:50.379955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:50.381516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:50.439463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 839/1658] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:50.442176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:50.444693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:50.552828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60372af-1bbe-43f6-b9b2-f7092c0de00f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:50.554029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931e0da-4ca6-448f-b3f8-d43491fec5ed service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:50.561538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:50.562568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:50.562982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:50.563301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:50.564613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:50.565465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:50.565601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:50.565767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:50.579420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:50.582504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:50.632600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:50.632600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:50.640735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 840/1659] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:50.646557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 820/1660] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:52.781738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:52.787197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:52.824928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca85e98-d5b3-4ca3-86e8-f05db459b148 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:52.837374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:52.838163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:52.838452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:52.838790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] PATH_INFO: `/v3/projects/f37b888f1a584296a90d6dec62dbb664` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:52.842036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f37b888f1a584296a90d6dec62dbb664)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:52.844005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:52.917158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:52.920424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:52.923724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:52.955227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927fa5e-537a-4080-a6ae-e4b959650b0a service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:52.983558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f37b888f1a584296a90d6dec62dbb664', 'request_id': 'req-92889894-9f96-4b71-9330-3ccdfa4cf138'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:52.993188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f37b888f1a584296a90d6dec62dbb664', 'request_id': 'req-92889894-9f96-4b71-9330-3ccdfa4cf138'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.006439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f37b888f1a584296a90d6dec62dbb664', 'request_id': 'req-92889894-9f96-4b71-9330-3ccdfa4cf138'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.050665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] The token cache is being invalidate because project f37b888f1a584296a90d6dec62dbb664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:13:53.055637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.061105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.065315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 841/1661] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:13:52 2022] DELETE /identity/v3/projects/f37b888f1a584296a90d6dec62dbb664 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:53.069491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:53.075932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:53.126792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:53.127802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:53.128232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:53.128694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] PATH_INFO: `/v3/projects/e650ca5f46214650b89cfba0e33bd957` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:53.130168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=e650ca5f46214650b89cfba0e33bd957)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:53.131514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 19:13:53.131514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Aug 09 19:13:53.131514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: Aug 09 19:13:53.133164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:53.284890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92889894-9f96-4b71-9330-3ccdfa4cf138 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:53.289096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e650ca5f46214650b89cfba0e33bd957', 'request_id': 'req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.294179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:53.311933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e650ca5f46214650b89cfba0e33bd957', 'request_id': 'req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.333922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e650ca5f46214650b89cfba0e33bd957', 'request_id': 'req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.375010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] The token cache is being invalidate because project e650ca5f46214650b89cfba0e33bd957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:13:53.375679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.376854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.382052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 821/1662] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:13:52 2022] DELETE /identity/v3/projects/e650ca5f46214650b89cfba0e33bd957 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:53.386728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:53.388533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:53.479451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:53.480117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:53.480394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:53.480688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] PATH_INFO: `/v3/users/7f85b5b3e27642fda3fabac620e88b8e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:53.491806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f85b5b3e27642fda3fabac620e88b8e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:53.494996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:53.550998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1d644-8fcc-47eb-a673-2b7aad7aa4ff admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:53.564697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:53.565462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:53.566530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:53.567714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] PATH_INFO: `/v3/users/b9a33353b1db4adc816dd83fe140fb7a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:53.569830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a33353b1db4adc816dd83fe140fb7a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:53.574954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:53.688369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f85b5b3e27642fda3fabac620e88b8e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.705325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f85b5b3e27642fda3fabac620e88b8e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.713899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:53.723637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a33353b1db4adc816dd83fe140fb7a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.731930 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f85b5b3e27642fda3fabac620e88b8e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.732640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f85b5b3e27642fda3fabac620e88b8e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.733262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-01b9c90e-35f6-4522-a680-64d954d8c360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f85b5b3e27642fda3fabac620e88b8e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.755210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a33353b1db4adc816dd83fe140fb7a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.782370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 842/1663] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:13:53 2022] DELETE /identity/v3/users/7f85b5b3e27642fda3fabac620e88b8e => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:53.782370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c505-7a95-41bd-bf53-60280ab56f7b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:53.782370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c505-7a95-41bd-bf53-60280ab56f7b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:53.782370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c505-7a95-41bd-bf53-60280ab56f7b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:53.789083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a33353b1db4adc816dd83fe140fb7a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.790977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a33353b1db4adc816dd83fe140fb7a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.792023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 843/1664] 10.222.0.70 () {54 vars in 962 bytes} [Tue Aug 9 19:13:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:13:53.793445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dcb52-73ee-4914-86d6-b5040b57457f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:53.793727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dcb52-73ee-4914-86d6-b5040b57457f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:53.794004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dcb52-73ee-4914-86d6-b5040b57457f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:53.794531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d6682f-2250-4ca7-97a4-d75680696dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a33353b1db4adc816dd83fe140fb7a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:53.852700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 822/1665] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:13:53 2022] DELETE /identity/v3/users/b9a33353b1db4adc816dd83fe140fb7a => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:53.857120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ee8dcb52-73ee-4914-86d6-b5040b57457f None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:53.858367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ee8dcb52-73ee-4914-86d6-b5040b57457f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:54.043910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06055567-08bd-4324-9bac-68c233862b16 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:54.044395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06055567-08bd-4324-9bac-68c233862b16 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:54.044505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06055567-08bd-4324-9bac-68c233862b16 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:54.046000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 823/1666] 10.222.0.70 () {54 vars in 912 bytes} [Tue Aug 9 19:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 19:13:54.049793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 844/1667] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Aug 9 19:13:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:54.053771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d8bfc-901a-466f-823f-ad09a3d67ac6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:54.054066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d8bfc-901a-466f-823f-ad09a3d67ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:54.054335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d8bfc-901a-466f-823f-ad09a3d67ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:54.083164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-cc3d8bfc-901a-466f-823f-ad09a3d67ac6 None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:54.084736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cc3d8bfc-901a-466f-823f-ad09a3d67ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:54.099818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 824/1668] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:13:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:54.116211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dcb52-73ee-4914-86d6-b5040b57457f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:54.118424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ee8dcb52-73ee-4914-86d6-b5040b57457f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:54.243674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dcb52-73ee-4914-86d6-b5040b57457f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:54.248459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b653c-7eb8-4529-80a3-0c53b0a0be5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:54.249109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b653c-7eb8-4529-80a3-0c53b0a0be5f service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:54.249455 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b653c-7eb8-4529-80a3-0c53b0a0be5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:54.249729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b653c-7eb8-4529-80a3-0c53b0a0be5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:54.275672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b653c-7eb8-4529-80a3-0c53b0a0be5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:54.277200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b653c-7eb8-4529-80a3-0c53b0a0be5f service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:54.327573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 845/1669] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:13:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:13:55.143207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d8bfc-901a-466f-823f-ad09a3d67ac6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:55.145415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cc3d8bfc-901a-466f-823f-ad09a3d67ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:55.276192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:55.299429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d8bfc-901a-466f-823f-ad09a3d67ac6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:55.312421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:55.314911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:55.315200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:55.315475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] PATH_INFO: `/v3/projects/984d23ece00f4aa8bf257331db874024` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:55.316683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] RBAC: Authorizing `identity:delete_project(project_id=984d23ece00f4aa8bf257331db874024)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:55.318615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:55.465286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984d23ece00f4aa8bf257331db874024', 'request_id': 'req-519cf6cb-5e4f-450d-89b8-957b1898982e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:55.471620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984d23ece00f4aa8bf257331db874024', 'request_id': 'req-519cf6cb-5e4f-450d-89b8-957b1898982e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:55.490955 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984d23ece00f4aa8bf257331db874024', 'request_id': 'req-519cf6cb-5e4f-450d-89b8-957b1898982e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:55.518792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] The token cache is being invalidate because project 984d23ece00f4aa8bf257331db874024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:13:55.521755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:55.526301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-519cf6cb-5e4f-450d-89b8-957b1898982e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:55.533105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 825/1670] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:13:55 2022] DELETE /identity/v3/projects/984d23ece00f4aa8bf257331db874024 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:55.536401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd07e4-61f9-4553-9529-1cf3f7f7f39e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:55.539278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd07e4-61f9-4553-9529-1cf3f7f7f39e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:55.539987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd07e4-61f9-4553-9529-1cf3f7f7f39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:55.606882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cabd07e4-61f9-4553-9529-1cf3f7f7f39e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:55.608545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cabd07e4-61f9-4553-9529-1cf3f7f7f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:55.774537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ad023-ab58-439b-931d-86fef7760092 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:55.775076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ad023-ab58-439b-931d-86fef7760092 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:55.775524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ad023-ab58-439b-931d-86fef7760092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:55.783541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 846/1671] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:55.811087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd07e4-61f9-4553-9529-1cf3f7f7f39e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:55.812660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cabd07e4-61f9-4553-9529-1cf3f7f7f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:55.829774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd07e4-61f9-4553-9529-1cf3f7f7f39e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:55.846940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:55.847777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:55.848113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:55.848393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] PATH_INFO: `/v3/projects/425c8090bcf3423ca3a9984ff8c61a60` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:55.850483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=425c8090bcf3423ca3a9984ff8c61a60)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:55.860603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:55.897222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:55.905494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1d4ad023-ab58-439b-931d-86fef7760092 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:55.910259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1d4ad023-ab58-439b-931d-86fef7760092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:55.958761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 826/1672] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:13:55.964121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ad023-ab58-439b-931d-86fef7760092 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.006690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ad023-ab58-439b-931d-86fef7760092 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:56.038065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:56.042016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.045263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.047342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.049282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:56.049926 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:56.053818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:56.117175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 827/1673] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:56.126802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.141830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425c8090bcf3423ca3a9984ff8c61a60', 'request_id': 'req-3964cfa1-a27a-4843-8c04-876183e69e13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:56.188433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf00522-fdf6-47c0-8836-65a1786400ba admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:56.193825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425c8090bcf3423ca3a9984ff8c61a60', 'request_id': 'req-3964cfa1-a27a-4843-8c04-876183e69e13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:56.217434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425c8090bcf3423ca3a9984ff8c61a60', 'request_id': 'req-3964cfa1-a27a-4843-8c04-876183e69e13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:56.238731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] The token cache is being invalidate because project 425c8090bcf3423ca3a9984ff8c61a60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:13:56.239035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:56.239726 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3964cfa1-a27a-4843-8c04-876183e69e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:13:56.242338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 847/1674] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:13:55 2022] DELETE /identity/v3/projects/425c8090bcf3423ca3a9984ff8c61a60 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:13:56.247958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6ee2d-2f10-4601-b276-b8e20236108d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.248671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6ee2d-2f10-4601-b276-b8e20236108d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.248952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6ee2d-2f10-4601-b276-b8e20236108d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.289035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:56.290157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d3c6ee2d-2f10-4601-b276-b8e20236108d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:56.290556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.291073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.291429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.291638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d3c6ee2d-2f10-4601-b276-b8e20236108d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:56.292227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:56.297293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:56.314936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 848/1675] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:56.322649 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899482ef-3176-4b71-b3d2-e9807da89820 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.322933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899482ef-3176-4b71-b3d2-e9807da89820 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.323206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899482ef-3176-4b71-b3d2-e9807da89820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.341424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 828/1676] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:56 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:56.349380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.353458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:56.396609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-899482ef-3176-4b71-b3d2-e9807da89820 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:56.398131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-899482ef-3176-4b71-b3d2-e9807da89820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:56.426878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:56.457783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e55ab-e301-4ba1-a176-dc4546253ce6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:56.465836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb4eb4-74bf-4146-aca4-ed6115dbafdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:56.466992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb4eb4-74bf-4146-aca4-ed6115dbafdd admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.467479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb4eb4-74bf-4146-aca4-ed6115dbafdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.468108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb4eb4-74bf-4146-aca4-ed6115dbafdd admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.469076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb4eb4-74bf-4146-aca4-ed6115dbafdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:56.471397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb4eb4-74bf-4146-aca4-ed6115dbafdd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:56.502741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 829/1677] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:56 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:56.506789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb4eb4-74bf-4146-aca4-ed6115dbafdd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.522873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb4eb4-74bf-4146-aca4-ed6115dbafdd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:56.535699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fbcaf-b982-4ac5-8d46-255e8f1396d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:56.538896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fbcaf-b982-4ac5-8d46-255e8f1396d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.545714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 849/1678] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:56.547714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899482ef-3176-4b71-b3d2-e9807da89820 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.550133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fbcaf-b982-4ac5-8d46-255e8f1396d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.550779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fbcaf-b982-4ac5-8d46-255e8f1396d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.551792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fbcaf-b982-4ac5-8d46-255e8f1396d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:56.553436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fbcaf-b982-4ac5-8d46-255e8f1396d7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:56.566536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 830/1679] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:56.567660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899482ef-3176-4b71-b3d2-e9807da89820 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:56.570534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fbcaf-b982-4ac5-8d46-255e8f1396d7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.578223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:56.579536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.580860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.581822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.589232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:13:56.589807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:56.594097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:56.598745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fbcaf-b982-4ac5-8d46-255e8f1396d7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:56.612582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97303a-34fd-4799-a057-62b50165b523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:56.613389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97303a-34fd-4799-a057-62b50165b523 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.613845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97303a-34fd-4799-a057-62b50165b523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.617736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97303a-34fd-4799-a057-62b50165b523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.618643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97303a-34fd-4799-a057-62b50165b523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:56.623828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97303a-34fd-4799-a057-62b50165b523 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:56.625570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 850/1680] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:13:56.627584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.647172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d49ad1-f439-47db-999f-77ad48bed6c9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:56.648387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 831/1681] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:56.653648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97303a-34fd-4799-a057-62b50165b523 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.662511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa152280-0906-42dd-b50b-09c083bc8e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:56.663802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa152280-0906-42dd-b50b-09c083bc8e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.664875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa152280-0906-42dd-b50b-09c083bc8e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.667815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa152280-0906-42dd-b50b-09c083bc8e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.668830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa152280-0906-42dd-b50b-09c083bc8e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:56.673478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97303a-34fd-4799-a057-62b50165b523 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:56.675650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa152280-0906-42dd-b50b-09c083bc8e92 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:56.689765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87bea4-6801-4ded-879b-a693d533c34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:56.693853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87bea4-6801-4ded-879b-a693d533c34d admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.695964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87bea4-6801-4ded-879b-a693d533c34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.697190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87bea4-6801-4ded-879b-a693d533c34d admin admin] PATH_INFO: `/v3/projects/50f6078d20984fd5adb321995df36de1/users/373415b60f244ed2a482dd4ac2af344f/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.725042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87bea4-6801-4ded-879b-a693d533c34d admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f6078d20984fd5adb321995df36de1, user_id=373415b60f244ed2a482dd4ac2af344f, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:56.734991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87bea4-6801-4ded-879b-a693d533c34d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:56.739539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 851/1682] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:56 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:56.743627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa152280-0906-42dd-b50b-09c083bc8e92 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:56.776156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 832/1683] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:56 2022] PUT /identity/v3/projects/50f6078d20984fd5adb321995df36de1/users/373415b60f244ed2a482dd4ac2af344f/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:13:56.777724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fda73-3b78-40f6-85b0-2aff98527111 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:56.780174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fda73-3b78-40f6-85b0-2aff98527111 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:56.780493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fda73-3b78-40f6-85b0-2aff98527111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:56.861269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5e1fda73-3b78-40f6-85b0-2aff98527111 None None] MFA Rules not processed for user `373415b60f244ed2a482dd4ac2af344f`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:56.864249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5e1fda73-3b78-40f6-85b0-2aff98527111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:56.889627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:56.937736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa152280-0906-42dd-b50b-09c083bc8e92 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.079769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 833/1684] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:57.084186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fda73-3b78-40f6-85b0-2aff98527111 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:57.230372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30916ca4-d6f2-4924-975a-2516bdf0d2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.234382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30916ca4-d6f2-4924-975a-2516bdf0d2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.237597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30916ca4-d6f2-4924-975a-2516bdf0d2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.239337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30916ca4-d6f2-4924-975a-2516bdf0d2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.245307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30916ca4-d6f2-4924-975a-2516bdf0d2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.260815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30916ca4-d6f2-4924-975a-2516bdf0d2d6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.295384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fda73-3b78-40f6-85b0-2aff98527111 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.313938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0a91e-3be0-4ead-8832-4d32e136c4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.314159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a91e-3be0-4ead-8832-4d32e136c4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.314432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a91e-3be0-4ead-8832-4d32e136c4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.314714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a91e-3be0-4ead-8832-4d32e136c4b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.319203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0a91e-3be0-4ead-8832-4d32e136c4b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.325557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0a91e-3be0-4ead-8832-4d32e136c4b2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.350287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 852/1685] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:56 2022] POST /identity/v3/users => generated 303 bytes in 607 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:57.359934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30916ca4-d6f2-4924-975a-2516bdf0d2d6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:57.402247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 834/1686] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:57 2022] POST /identity/v3/projects => generated 379 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:57.403189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0a91e-3be0-4ead-8832-4d32e136c4b2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:57.411954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:57.412615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30916ca4-d6f2-4924-975a-2516bdf0d2d6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.497651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0a91e-3be0-4ead-8832-4d32e136c4b2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.504072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272475e-2225-4b0c-b87a-2c7657207147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.504758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272475e-2225-4b0c-b87a-2c7657207147 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.505037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272475e-2225-4b0c-b87a-2c7657207147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.505315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272475e-2225-4b0c-b87a-2c7657207147 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.505949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272475e-2225-4b0c-b87a-2c7657207147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.508800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272475e-2225-4b0c-b87a-2c7657207147 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.547371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 835/1687] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:57 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:57.549669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272475e-2225-4b0c-b87a-2c7657207147 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:57.563772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272475e-2225-4b0c-b87a-2c7657207147 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.585331 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709189-d483-4b63-85d2-c9c758b8876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.585891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709189-d483-4b63-85d2-c9c758b8876c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.586171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709189-d483-4b63-85d2-c9c758b8876c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.586450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709189-d483-4b63-85d2-c9c758b8876c admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.587124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0709189-d483-4b63-85d2-c9c758b8876c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.588802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0709189-d483-4b63-85d2-c9c758b8876c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.602303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 836/1688] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:57.606684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709189-d483-4b63-85d2-c9c758b8876c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:57.607256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b62737-278d-45e6-b600-bd8fcebc7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.609040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b62737-278d-45e6-b600-bd8fcebc7afe admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.609685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b62737-278d-45e6-b600-bd8fcebc7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.610205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b62737-278d-45e6-b600-bd8fcebc7afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.611510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b62737-278d-45e6-b600-bd8fcebc7afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.613568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b62737-278d-45e6-b600-bd8fcebc7afe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.627983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 853/1689] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:57.631670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709189-d483-4b63-85d2-c9c758b8876c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.632976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b62737-278d-45e6-b600-bd8fcebc7afe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:57.646456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b614b-6fd6-44c7-bb57-f4aca4f7023c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.647816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b614b-6fd6-44c7-bb57-f4aca4f7023c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.650359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b614b-6fd6-44c7-bb57-f4aca4f7023c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.651023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b614b-6fd6-44c7-bb57-f4aca4f7023c admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.652343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b614b-6fd6-44c7-bb57-f4aca4f7023c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.656646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b614b-6fd6-44c7-bb57-f4aca4f7023c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.657483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b62737-278d-45e6-b600-bd8fcebc7afe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.670319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804ce49-c1b9-41cf-974b-8b3dd3e6e01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.671425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ce49-c1b9-41cf-974b-8b3dd3e6e01b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.671827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ce49-c1b9-41cf-974b-8b3dd3e6e01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.672942 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ce49-c1b9-41cf-974b-8b3dd3e6e01b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.674150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804ce49-c1b9-41cf-974b-8b3dd3e6e01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.677993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804ce49-c1b9-41cf-974b-8b3dd3e6e01b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.687989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 837/1690] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:57.689293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b614b-6fd6-44c7-bb57-f4aca4f7023c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:57.709810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 854/1691] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:57.710893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b614b-6fd6-44c7-bb57-f4aca4f7023c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.712826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804ce49-c1b9-41cf-974b-8b3dd3e6e01b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:57.723041 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562c361-17cf-46aa-a64f-66e90a505183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.724195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562c361-17cf-46aa-a64f-66e90a505183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.724550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562c361-17cf-46aa-a64f-66e90a505183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.724893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562c361-17cf-46aa-a64f-66e90a505183 admin admin] PATH_INFO: `/v3/projects/73dba1bf84484e10b1fc5f7135b08985/users/ee9221c2d127455fad94498069b05ede/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.734810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804ce49-c1b9-41cf-974b-8b3dd3e6e01b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:57.751458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89f15b-6013-4a2b-b513-7a50afda86a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:57.753855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f15b-6013-4a2b-b513-7a50afda86a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.754151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f15b-6013-4a2b-b513-7a50afda86a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.754518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89f15b-6013-4a2b-b513-7a50afda86a8 admin admin] PATH_INFO: `/v3/projects/eae32c33b0744829bb453166d0240d8e/users/07f80bede751418c8ab5c26faba0bc95/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.771843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562c361-17cf-46aa-a64f-66e90a505183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73dba1bf84484e10b1fc5f7135b08985, user_id=ee9221c2d127455fad94498069b05ede, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.786178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562c361-17cf-46aa-a64f-66e90a505183 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.795487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89f15b-6013-4a2b-b513-7a50afda86a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae32c33b0744829bb453166d0240d8e, user_id=07f80bede751418c8ab5c26faba0bc95, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:57.801001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89f15b-6013-4a2b-b513-7a50afda86a8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:57.825381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 838/1692] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:57 2022] PUT /identity/v3/projects/73dba1bf84484e10b1fc5f7135b08985/users/ee9221c2d127455fad94498069b05ede/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:57.827576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4430b-205c-4844-b99a-d8beea97c84e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.829167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4430b-205c-4844-b99a-d8beea97c84e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.829374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4430b-205c-4844-b99a-d8beea97c84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.838798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 855/1693] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:57 2022] PUT /identity/v3/projects/eae32c33b0744829bb453166d0240d8e/users/07f80bede751418c8ab5c26faba0bc95/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:57.852363 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b5f3-9096-441c-95e8-3376e10386b8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:57.852812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b5f3-9096-441c-95e8-3376e10386b8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:57.853151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b5f3-9096-441c-95e8-3376e10386b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:57.925046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-72a4430b-205c-4844-b99a-d8beea97c84e None None] MFA Rules not processed for user `ee9221c2d127455fad94498069b05ede`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:57.933011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-72a4430b-205c-4844-b99a-d8beea97c84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:57.951254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-61a7b5f3-9096-441c-95e8-3376e10386b8 None None] MFA Rules not processed for user `07f80bede751418c8ab5c26faba0bc95`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:57.957238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-61a7b5f3-9096-441c-95e8-3376e10386b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:58.089991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:58.159261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 839/1694] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:58.163783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4430b-205c-4844-b99a-d8beea97c84e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:58.190136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 856/1695] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:58.195462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7b5f3-9096-441c-95e8-3376e10386b8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:58.288899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:58.373935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4430b-205c-4844-b99a-d8beea97c84e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:58.405115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14e60-7929-4f62-afe5-5afe09696432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:58.405732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14e60-7929-4f62-afe5-5afe09696432 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:58.406025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14e60-7929-4f62-afe5-5afe09696432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:58.406274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14e60-7929-4f62-afe5-5afe09696432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:58.406901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e14e60-7929-4f62-afe5-5afe09696432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:58.412523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e14e60-7929-4f62-afe5-5afe09696432 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:58.418045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7b5f3-9096-441c-95e8-3376e10386b8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:58.429170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce705e-a692-45ef-ae3a-db69d27af5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:58.431243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce705e-a692-45ef-ae3a-db69d27af5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:58.431630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce705e-a692-45ef-ae3a-db69d27af5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:58.431913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce705e-a692-45ef-ae3a-db69d27af5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:58.435421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce705e-a692-45ef-ae3a-db69d27af5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:58.445678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce705e-a692-45ef-ae3a-db69d27af5a1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:58.504649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 840/1696] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:58 2022] POST /identity/v3/projects => generated 379 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:13:58.508613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14e60-7929-4f62-afe5-5afe09696432 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:58.559753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 857/1697] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:13:58 2022] POST /identity/v3/projects => generated 407 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:13:58.563697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce705e-a692-45ef-ae3a-db69d27af5a1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:58.639262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14e60-7929-4f62-afe5-5afe09696432 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:58.777747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce705e-a692-45ef-ae3a-db69d27af5a1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:58.795928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6e380-8d51-48be-9b0b-ddfacf2d63cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:58.796830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e380-8d51-48be-9b0b-ddfacf2d63cc admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:58.797129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e380-8d51-48be-9b0b-ddfacf2d63cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:58.797412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e380-8d51-48be-9b0b-ddfacf2d63cc admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:58.798076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6e380-8d51-48be-9b0b-ddfacf2d63cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:58.800108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6e380-8d51-48be-9b0b-ddfacf2d63cc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:58.847239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd973d4c-4eb8-46db-8c15-f6d0c875918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:58.848400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd973d4c-4eb8-46db-8c15-f6d0c875918c admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:58.848776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd973d4c-4eb8-46db-8c15-f6d0c875918c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:58.849034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd973d4c-4eb8-46db-8c15-f6d0c875918c admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:58.849686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd973d4c-4eb8-46db-8c15-f6d0c875918c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:58.852573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd973d4c-4eb8-46db-8c15-f6d0c875918c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:58.892020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 858/1698] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:58 2022] POST /identity/v3/users => generated 310 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:13:58.894277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6e380-8d51-48be-9b0b-ddfacf2d63cc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:58.935220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 841/1699] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:13:58 2022] POST /identity/v3/users => generated 296 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:13:58.938146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd973d4c-4eb8-46db-8c15-f6d0c875918c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:58.950440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6e380-8d51-48be-9b0b-ddfacf2d63cc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:58.975653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36f1a7-e3c6-4290-a77b-231e6875eaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:58.979147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36f1a7-e3c6-4290-a77b-231e6875eaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:58.979437 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36f1a7-e3c6-4290-a77b-231e6875eaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:58.980673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36f1a7-e3c6-4290-a77b-231e6875eaf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:58.984256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd973d4c-4eb8-46db-8c15-f6d0c875918c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:58.995155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36f1a7-e3c6-4290-a77b-231e6875eaf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:59.000302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36f1a7-e3c6-4290-a77b-231e6875eaf3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:59.007014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf2031-a2ff-488d-9634-61382d18dc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:59.007654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf2031-a2ff-488d-9634-61382d18dc80 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:59.008039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf2031-a2ff-488d-9634-61382d18dc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:59.008551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf2031-a2ff-488d-9634-61382d18dc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:59.009502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf2031-a2ff-488d-9634-61382d18dc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:59.012755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 859/1700] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:59.016894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36f1a7-e3c6-4290-a77b-231e6875eaf3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:59.024486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf2031-a2ff-488d-9634-61382d18dc80 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:59.043147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36f1a7-e3c6-4290-a77b-231e6875eaf3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:59.051722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8ff16-1688-4bdd-8d33-c65d1c4a8e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:59.052702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ff16-1688-4bdd-8d33-c65d1c4a8e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:59.053093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ff16-1688-4bdd-8d33-c65d1c4a8e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:59.053439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ff16-1688-4bdd-8d33-c65d1c4a8e39 admin admin] PATH_INFO: `/v3/projects/15e0d0eb463a4bbc83ffdca351bbf762/users/1122c364ab1141a7a74bca56efc94892/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:59.058966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 842/1701] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:13:59.062947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf2031-a2ff-488d-9634-61382d18dc80 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:59.075753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8ff16-1688-4bdd-8d33-c65d1c4a8e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e0d0eb463a4bbc83ffdca351bbf762, user_id=1122c364ab1141a7a74bca56efc94892, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:59.079660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8ff16-1688-4bdd-8d33-c65d1c4a8e39 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:59.108480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 860/1702] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:59 2022] PUT /identity/v3/projects/15e0d0eb463a4bbc83ffdca351bbf762/users/1122c364ab1141a7a74bca56efc94892/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:59.113425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8ff16-1688-4bdd-8d33-c65d1c4a8e39 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:59.113996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf2031-a2ff-488d-9634-61382d18dc80 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:59.208249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:59.304476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8ff16-1688-4bdd-8d33-c65d1c4a8e39 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:59.307197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:13:59.328920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f27f9-a706-410f-b71f-e15f1d93cede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:59.329856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f27f9-a706-410f-b71f-e15f1d93cede admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:59.330339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f27f9-a706-410f-b71f-e15f1d93cede admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:59.330840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f27f9-a706-410f-b71f-e15f1d93cede admin admin] PATH_INFO: `/v3/projects/52afcbdaf190427cb4b3cae73bfbdbef/users/4e0dacd71b784c26a0a30f5991534ef4/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:59.333700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9c0a3-8c83-4687-b88c-302c4c5b21d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:59.335436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9c0a3-8c83-4687-b88c-302c4c5b21d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:59.335886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9c0a3-8c83-4687-b88c-302c4c5b21d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:59.337373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9c0a3-8c83-4687-b88c-302c4c5b21d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:59.339196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9c0a3-8c83-4687-b88c-302c4c5b21d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:59.342688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9c0a3-8c83-4687-b88c-302c4c5b21d0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:59.376868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f27f9-a706-410f-b71f-e15f1d93cede admin admin] RBAC: Authorizing `identity:create_grant(project_id=52afcbdaf190427cb4b3cae73bfbdbef, user_id=4e0dacd71b784c26a0a30f5991534ef4, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:59.379422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 861/1703] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:13:59.381588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f27f9-a706-410f-b71f-e15f1d93cede admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:59.382113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900335d-2a71-4050-87f8-bfb00339dac3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:59.382577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900335d-2a71-4050-87f8-bfb00339dac3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:59.384362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900335d-2a71-4050-87f8-bfb00339dac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:59.423248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 843/1704] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:59 2022] PUT /identity/v3/projects/52afcbdaf190427cb4b3cae73bfbdbef/users/4e0dacd71b784c26a0a30f5991534ef4/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:13:59.424275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f27f9-a706-410f-b71f-e15f1d93cede admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:59.445144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d900335d-2a71-4050-87f8-bfb00339dac3 None None] MFA Rules not processed for user `e9ad80a549b14b25ac71cd96832a74e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:59.449060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d900335d-2a71-4050-87f8-bfb00339dac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:59.561106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f27f9-a706-410f-b71f-e15f1d93cede admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:13:59.614823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d943f-220c-43ee-9e86-bd81d6d9cc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:13:59.619455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d943f-220c-43ee-9e86-bd81d6d9cc38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:59.619751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d943f-220c-43ee-9e86-bd81d6d9cc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:59.623476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d943f-220c-43ee-9e86-bd81d6d9cc38 admin admin] PATH_INFO: `/v3/projects/15e0d0eb463a4bbc83ffdca351bbf762/users/1122c364ab1141a7a74bca56efc94892/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:59.631457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d943f-220c-43ee-9e86-bd81d6d9cc38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e0d0eb463a4bbc83ffdca351bbf762, user_id=1122c364ab1141a7a74bca56efc94892, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:13:59.636254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d943f-220c-43ee-9e86-bd81d6d9cc38 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:13:59.668510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 844/1705] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:13:59 2022] PUT /identity/v3/projects/15e0d0eb463a4bbc83ffdca351bbf762/users/1122c364ab1141a7a74bca56efc94892/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:13:59.669626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d943f-220c-43ee-9e86-bd81d6d9cc38 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:13:59.690798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 862/1706] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:13:59.692273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe70f47-0d68-4aa3-8c91-90618616a883 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:13:59.698462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe70f47-0d68-4aa3-8c91-90618616a883 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:13:59.700822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe70f47-0d68-4aa3-8c91-90618616a883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:13:59.828540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dfe70f47-0d68-4aa3-8c91-90618616a883 None None] MFA Rules not processed for user `8755b16a18734175b691f891351d5116`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:13:59.830790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dfe70f47-0d68-4aa3-8c91-90618616a883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:13:59.989560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d943f-220c-43ee-9e86-bd81d6d9cc38 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:00.008826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6034f-1a53-4d7c-b10a-942f87505391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:00.010452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6034f-1a53-4d7c-b10a-942f87505391 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.011021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6034f-1a53-4d7c-b10a-942f87505391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.011639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6034f-1a53-4d7c-b10a-942f87505391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.012741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6034f-1a53-4d7c-b10a-942f87505391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:00.014724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6034f-1a53-4d7c-b10a-942f87505391 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:00.059677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 845/1707] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:00.063569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48983155-7621-48ee-a072-48b3fbea4694 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.064051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48983155-7621-48ee-a072-48b3fbea4694 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.065828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48983155-7621-48ee-a072-48b3fbea4694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.115978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-48983155-7621-48ee-a072-48b3fbea4694 None None] MFA Rules not processed for user `1122c364ab1141a7a74bca56efc94892`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:00.119351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-48983155-7621-48ee-a072-48b3fbea4694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:00.144078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 863/1708] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:13:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:00.145693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe70f47-0d68-4aa3-8c91-90618616a883 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:00.147275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dfe70f47-0d68-4aa3-8c91-90618616a883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:00.234484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:00.255363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:00.299094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 846/1709] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:00.300672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63083260-5ef6-443d-968b-3377479b4ec5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.300961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63083260-5ef6-443d-968b-3377479b4ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.301242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63083260-5ef6-443d-968b-3377479b4ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.319190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe70f47-0d68-4aa3-8c91-90618616a883 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:00.337931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bea578-6817-43a0-9924-e067716ac425 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:00.338512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea578-6817-43a0-9924-e067716ac425 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.338812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea578-6817-43a0-9924-e067716ac425 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.339064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea578-6817-43a0-9924-e067716ac425 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.385238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-63083260-5ef6-443d-968b-3377479b4ec5 None None] MFA Rules not processed for user `5464533ccfb74094bf613bcd51c55ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:00.387171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-63083260-5ef6-443d-968b-3377479b4ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:00.472665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bea578-6817-43a0-9924-e067716ac425 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:00.475554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bea578-6817-43a0-9924-e067716ac425 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:00.501066 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 864/1710] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:00.501990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bea578-6817-43a0-9924-e067716ac425 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:00.519806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 847/1711] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:00.521345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bea578-6817-43a0-9924-e067716ac425 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:00.523456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75768b0-1dac-4805-b332-6092bc6d39dd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.524988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75768b0-1dac-4805-b332-6092bc6d39dd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.525341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75768b0-1dac-4805-b332-6092bc6d39dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.527368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:00.528271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.528734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.529354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] PATH_INFO: `/v3/projects/52afcbdaf190427cb4b3cae73bfbdbef/users/4e0dacd71b784c26a0a30f5991534ef4/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.532059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=52afcbdaf190427cb4b3cae73bfbdbef, user_id=4e0dacd71b784c26a0a30f5991534ef4, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:00.535110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:00.548502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 865/1712] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:00 2022] PUT /identity/v3/projects/52afcbdaf190427cb4b3cae73bfbdbef/users/4e0dacd71b784c26a0a30f5991534ef4/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:00.549958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:00.553198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:00.585066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f75768b0-1dac-4805-b332-6092bc6d39dd None None] MFA Rules not processed for user `8755b16a18734175b691f891351d5116`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:00.598812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f75768b0-1dac-4805-b332-6092bc6d39dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:00.679028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c6c5f-c800-41af-a17e-36f53ff8f4dd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:00.685362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21fa7b-7cfd-4d67-a957-54d6cb97356b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:00.686088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21fa7b-7cfd-4d67-a957-54d6cb97356b service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.686432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21fa7b-7cfd-4d67-a957-54d6cb97356b service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.686866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21fa7b-7cfd-4d67-a957-54d6cb97356b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.737266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 848/1713] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:00.739004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e34634-a571-484c-855c-01fd37621693 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.740945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e34634-a571-484c-855c-01fd37621693 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.749062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e34634-a571-484c-855c-01fd37621693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.803957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21fa7b-7cfd-4d67-a957-54d6cb97356b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:00.803957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21fa7b-7cfd-4d67-a957-54d6cb97356b service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:00.813216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 866/1714] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:00.814481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5df27-7ce1-4b91-871c-ddec7743a52d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:00.814821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5df27-7ce1-4b91-871c-ddec7743a52d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:00.815102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5df27-7ce1-4b91-871c-ddec7743a52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:00.815761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-82e34634-a571-484c-855c-01fd37621693 None None] MFA Rules not processed for user `5464533ccfb74094bf613bcd51c55ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:00.823499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-82e34634-a571-484c-855c-01fd37621693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:00.841548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:00.858756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-40d5df27-7ce1-4b91-871c-ddec7743a52d None None] MFA Rules not processed for user `4e0dacd71b784c26a0a30f5991534ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:00.860285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-40d5df27-7ce1-4b91-871c-ddec7743a52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:00.991130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 849/1715] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:00.995776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e34634-a571-484c-855c-01fd37621693 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:01.042012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 867/1716] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:01.109883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:01.246957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e34634-a571-484c-855c-01fd37621693 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:01.278123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:01.278123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:01.278123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:01.278123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:01.286783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5df27-7ce1-4b91-871c-ddec7743a52d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:01.299303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:01.299500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5df27-7ce1-4b91-871c-ddec7743a52d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:01.301224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:01.305504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552dbd7-2432-4006-987d-e5070523cfb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:01.308595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552dbd7-2432-4006-987d-e5070523cfb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:01.309063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552dbd7-2432-4006-987d-e5070523cfb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:01.309365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552dbd7-2432-4006-987d-e5070523cfb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:01.312641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 850/1717] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:00 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:01.313966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:01.315405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:01.335315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ccaff-2259-483d-a569-bb880cb49bae service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:01.344240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:01.345646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:01.345997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:01.346346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:01.357731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552dbd7-2432-4006-987d-e5070523cfb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:01.379396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552dbd7-2432-4006-987d-e5070523cfb6 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:01.382479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:01.385021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:01.406576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 868/1718] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:01.422542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 851/1719] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:01.491170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552dbd7-2432-4006-987d-e5070523cfb6 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:01.505269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552dbd7-2432-4006-987d-e5070523cfb6 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:01.519154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f42746-7f2a-4fa8-90e3-b0f8df4e0b52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:01.519830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42746-7f2a-4fa8-90e3-b0f8df4e0b52 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:01.520116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42746-7f2a-4fa8-90e3-b0f8df4e0b52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:01.520394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f42746-7f2a-4fa8-90e3-b0f8df4e0b52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:01.659420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f42746-7f2a-4fa8-90e3-b0f8df4e0b52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:01.661015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f42746-7f2a-4fa8-90e3-b0f8df4e0b52 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:01.685293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 869/1720] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:01 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:01.770025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:01.772059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:01.799082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca954af-1cb9-4613-93fd-e3b003c0e10e service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:01.807306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343eadd-2b26-4f4b-b770-b631e6f682b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:01.809315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343eadd-2b26-4f4b-b770-b631e6f682b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:01.809793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343eadd-2b26-4f4b-b770-b631e6f682b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:01.810242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343eadd-2b26-4f4b-b770-b631e6f682b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:01.994888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343eadd-2b26-4f4b-b770-b631e6f682b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:02.000648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343eadd-2b26-4f4b-b770-b631e6f682b5 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:02.049222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 852/1721] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:02.346006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bce518-832c-4060-97ed-4fdfb2db8ff4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:02.346460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bce518-832c-4060-97ed-4fdfb2db8ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:02.346594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bce518-832c-4060-97ed-4fdfb2db8ff4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:02.348417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 870/1722] 10.222.0.70 () {54 vars in 912 bytes} [Tue Aug 9 19:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:14:02.361612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142fc46-8312-4c7c-a5ae-8ef6afe2bf65 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:02.362313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ddeeb-26a2-42e3-ac9b-7744c93abef1 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:02.362596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ddeeb-26a2-42e3-ac9b-7744c93abef1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:02.362872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ddeeb-26a2-42e3-ac9b-7744c93abef1 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:02.364403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 871/1723] 10.222.0.70 () {54 vars in 912 bytes} [Tue Aug 9 19:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:14:02.364529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142fc46-8312-4c7c-a5ae-8ef6afe2bf65 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:02.364710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142fc46-8312-4c7c-a5ae-8ef6afe2bf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:02.370866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e4fa8-ace5-4377-9517-568069114a06 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:02.371160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e4fa8-ace5-4377-9517-568069114a06 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:02.371445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e4fa8-ace5-4377-9517-568069114a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:02.506899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f142fc46-8312-4c7c-a5ae-8ef6afe2bf65 None None] MFA Rules not processed for user `31183ac0fdd642e6a1a7578899aa9fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:02.511447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f142fc46-8312-4c7c-a5ae-8ef6afe2bf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:02.515294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-287e4fa8-ace5-4377-9517-568069114a06 None None] MFA Rules not processed for user `31183ac0fdd642e6a1a7578899aa9fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:02.536022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-287e4fa8-ace5-4377-9517-568069114a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:02.720048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 853/1724] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:14:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:02.737426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142fc46-8312-4c7c-a5ae-8ef6afe2bf65 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:02.737609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f142fc46-8312-4c7c-a5ae-8ef6afe2bf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:02.742977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 872/1725] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:14:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:02.767875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142fc46-8312-4c7c-a5ae-8ef6afe2bf65 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:02.778624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473074ec-6657-486b-a7be-60a3ccf618e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:02.779243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473074ec-6657-486b-a7be-60a3ccf618e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:02.779533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473074ec-6657-486b-a7be-60a3ccf618e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:02.779811 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473074ec-6657-486b-a7be-60a3ccf618e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:02.784719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e4fa8-ace5-4377-9517-568069114a06 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:02.786637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-287e4fa8-ace5-4377-9517-568069114a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:02.806821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e4fa8-ace5-4377-9517-568069114a06 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:02.811314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473074ec-6657-486b-a7be-60a3ccf618e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:02.814005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473074ec-6657-486b-a7be-60a3ccf618e2 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:02.818818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d705ff-c9b7-4c7c-8118-d1573996abc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:02.819203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d705ff-c9b7-4c7c-8118-d1573996abc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:02.819458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d705ff-c9b7-4c7c-8118-d1573996abc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:02.819731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d705ff-c9b7-4c7c-8118-d1573996abc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:02.833240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 854/1726] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:02.834065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d705ff-c9b7-4c7c-8118-d1573996abc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:02.836477 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d705ff-c9b7-4c7c-8118-d1573996abc2 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:02.867444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 873/1727] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:03.542264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebb0b-6d87-4e82-af0b-6965cb7e7729 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:03.542760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebb0b-6d87-4e82-af0b-6965cb7e7729 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:03.542835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebb0b-6d87-4e82-af0b-6965cb7e7729 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:03.544149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 855/1728] 10.222.0.70 () {54 vars in 961 bytes} [Tue Aug 9 19:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:14:03.557138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f9f4-3afb-4f1a-8ff8-b12184142428 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:03.557600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f9f4-3afb-4f1a-8ff8-b12184142428 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:03.557670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f9f4-3afb-4f1a-8ff8-b12184142428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:03.567318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:03.626990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f5e2f9f4-3afb-4f1a-8ff8-b12184142428 None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:03.628243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f5e2f9f4-3afb-4f1a-8ff8-b12184142428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:03.858367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbebb0b-6d87-4e82-af0b-6965cb7e7729 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:03.870680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 874/1729] 10.222.0.70 () {58 vars in 1058 bytes} [Tue Aug 9 19:14:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:03.880225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbebb0b-6d87-4e82-af0b-6965cb7e7729 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:03.889822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2f9f4-3afb-4f1a-8ff8-b12184142428 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:03.891227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f5e2f9f4-3afb-4f1a-8ff8-b12184142428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:03.917003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a732ee-8c35-4423-a6c9-604e0646e222 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:03.917791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a732ee-8c35-4423-a6c9-604e0646e222 service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:03.918081 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a732ee-8c35-4423-a6c9-604e0646e222 service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:03.918358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a732ee-8c35-4423-a6c9-604e0646e222 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:03.923629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-38a732ee-8c35-4423-a6c9-604e0646e222 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:03.936072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2f9f4-3afb-4f1a-8ff8-b12184142428 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:03.959522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0b5da-81d0-4954-8537-eb3e81109741 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:03.963453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0b5da-81d0-4954-8537-eb3e81109741 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:03.966390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0b5da-81d0-4954-8537-eb3e81109741 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:03.966665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0b5da-81d0-4954-8537-eb3e81109741 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:03.966987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a732ee-8c35-4423-a6c9-604e0646e222 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:03.971187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a732ee-8c35-4423-a6c9-604e0646e222 service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:04.003167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 856/1730] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:04.007583 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0b5da-81d0-4954-8537-eb3e81109741 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:04.013235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0b5da-81d0-4954-8537-eb3e81109741 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:04.064218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 875/1731] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:04.134290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895445b-18bc-49d0-8223-fba0c081d924 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:04.134935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895445b-18bc-49d0-8223-fba0c081d924 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:04.135055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895445b-18bc-49d0-8223-fba0c081d924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:04.157238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:04.167365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7895445b-18bc-49d0-8223-fba0c081d924 None None] MFA Rules not processed for user `8755b16a18734175b691f891351d5116`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:04.169661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7895445b-18bc-49d0-8223-fba0c081d924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:04.184068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 857/1732] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:04.580750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dac2e-7adc-4ae5-9fd3-72898a0db105 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:04.582050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dac2e-7adc-4ae5-9fd3-72898a0db105 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:04.587320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dac2e-7adc-4ae5-9fd3-72898a0db105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:04.601999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895445b-18bc-49d0-8223-fba0c081d924 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:04.603648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7895445b-18bc-49d0-8223-fba0c081d924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:04.704908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-390dac2e-7adc-4ae5-9fd3-72898a0db105 None None] MFA Rules not processed for user `e9ad80a549b14b25ac71cd96832a74e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:04.708293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-390dac2e-7adc-4ae5-9fd3-72898a0db105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:04.911037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895445b-18bc-49d0-8223-fba0c081d924 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:04.925531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:04.927477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:04.927869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:04.928392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] PATH_INFO: `/v3/users/f9b152fcdf9c4427830eeaba76870aa1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:04.928848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 876/1733] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:04.930372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b152fcdf9c4427830eeaba76870aa1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:04.941551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:05.091562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b152fcdf9c4427830eeaba76870aa1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:05.123924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b152fcdf9c4427830eeaba76870aa1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:05.156073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b152fcdf9c4427830eeaba76870aa1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:05.157424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b152fcdf9c4427830eeaba76870aa1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:05.158849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b152fcdf9c4427830eeaba76870aa1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:05.220519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 858/1734] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:04 2022] DELETE /identity/v3/users/f9b152fcdf9c4427830eeaba76870aa1 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:05.223029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3f47-dc45-412b-8bf9-acd31b145c56 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:05.223323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3f47-dc45-412b-8bf9-acd31b145c56 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:05.225129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3f47-dc45-412b-8bf9-acd31b145c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:05.312219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cbba3f47-dc45-412b-8bf9-acd31b145c56 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:05.316385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cbba3f47-dc45-412b-8bf9-acd31b145c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:05.511241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 877/1735] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:05.545087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:05.546577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:05.705233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b45e-679a-4c3f-b917-ffacd78e04aa admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:05.717822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041265-4d22-4c2a-a286-d7db6d7d04e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:05.718668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041265-4d22-4c2a-a286-d7db6d7d04e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:05.720377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041265-4d22-4c2a-a286-d7db6d7d04e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:05.721072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041265-4d22-4c2a-a286-d7db6d7d04e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:05.745778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24041265-4d22-4c2a-a286-d7db6d7d04e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:05.747486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24041265-4d22-4c2a-a286-d7db6d7d04e3 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:05.763168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 859/1736] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:06.874981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3f47-dc45-412b-8bf9-acd31b145c56 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:06.876542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cbba3f47-dc45-412b-8bf9-acd31b145c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:06.936723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3f47-dc45-412b-8bf9-acd31b145c56 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:06.957297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:06.958748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:06.959149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:06.959589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] PATH_INFO: `/v3/projects/d2ce0ec52d0245b99b49738dc8f34f73` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:06.962094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ce0ec52d0245b99b49738dc8f34f73)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:06.965117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:07.018457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:07.060332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ce0ec52d0245b99b49738dc8f34f73', 'request_id': 'req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.069363 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ce0ec52d0245b99b49738dc8f34f73', 'request_id': 'req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.085406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ce0ec52d0245b99b49738dc8f34f73', 'request_id': 'req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.105763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] The token cache is being invalidate because project d2ce0ec52d0245b99b49738dc8f34f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:07.106232 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.107090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.109604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 878/1737] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:06 2022] DELETE /identity/v3/projects/d2ce0ec52d0245b99b49738dc8f34f73 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:07.113393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041265-4d22-4c2a-a286-d7db6d7d04e3 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:07.132108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041265-4d22-4c2a-a286-d7db6d7d04e3 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:07.203664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:07.318256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:07.318916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:07.319199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:07.319473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] PATH_INFO: `/v3/users/3d92be02836d4b889e531f646b7b5b6e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:07.322236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d92be02836d4b889e531f646b7b5b6e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:07.324202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:07.470744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d92be02836d4b889e531f646b7b5b6e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.497466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d92be02836d4b889e531f646b7b5b6e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.528491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d92be02836d4b889e531f646b7b5b6e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.532535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d92be02836d4b889e531f646b7b5b6e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.536298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d92be02836d4b889e531f646b7b5b6e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:07.628748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 860/1738] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:07 2022] DELETE /identity/v3/users/3d92be02836d4b889e531f646b7b5b6e => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:08.560910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:08.562725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:08.593458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9281c5b-6d6c-4c47-b7f4-fd54ec6ff679 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:08.619938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:08.622181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:08.723778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e228c-336f-44f7-94d3-763cdd0bcfb1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:08.726145 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:08.727540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:08.727825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:08.728100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] PATH_INFO: `/v3/projects/de3cf5b2a95a430c944d315eabf00d9f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:08.729315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de3cf5b2a95a430c944d315eabf00d9f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:08.731245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:08.731812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:08.732622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:08.732976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:08.733324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] PATH_INFO: `/v3/users/373415b60f244ed2a482dd4ac2af344f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:08.734542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=373415b60f244ed2a482dd4ac2af344f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:08.737366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:08.858030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:08.868582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '373415b60f244ed2a482dd4ac2af344f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:08.874681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de3cf5b2a95a430c944d315eabf00d9f', 'request_id': 'req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:08.904252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '373415b60f244ed2a482dd4ac2af344f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:08.912943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de3cf5b2a95a430c944d315eabf00d9f', 'request_id': 'req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:08.932194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '373415b60f244ed2a482dd4ac2af344f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:08.934740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '373415b60f244ed2a482dd4ac2af344f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:08.949786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de3cf5b2a95a430c944d315eabf00d9f', 'request_id': 'req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:08.951645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-71f28201-1b1a-4d3d-aa8e-1ff5a0b2aa8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '373415b60f244ed2a482dd4ac2af344f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:08.999817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] The token cache is being invalidate because project de3cf5b2a95a430c944d315eabf00d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:09.000138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:09.007078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92c702bf-b4d1-4c26-a8f9-c2c95e45d2c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:09.017084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 879/1739] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:08 2022] DELETE /identity/v3/projects/de3cf5b2a95a430c944d315eabf00d9f => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:09.023318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d06d24-f8a2-48c1-b14b-3f1f9e371031 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.023912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d06d24-f8a2-48c1-b14b-3f1f9e371031 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.024343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d06d24-f8a2-48c1-b14b-3f1f9e371031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.032730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 861/1740] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:08 2022] DELETE /identity/v3/users/373415b60f244ed2a482dd4ac2af344f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:09.038554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ad9c8-9530-4550-9edb-aea042fc271e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.042523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ad9c8-9530-4550-9edb-aea042fc271e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.042816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ad9c8-9530-4550-9edb-aea042fc271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.084717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f9d06d24-f8a2-48c1-b14b-3f1f9e371031 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:09.088700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f9d06d24-f8a2-48c1-b14b-3f1f9e371031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:09.095612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6b6ad9c8-9530-4550-9edb-aea042fc271e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:09.102108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6b6ad9c8-9530-4550-9edb-aea042fc271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:09.179270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:09.233939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 880/1741] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:09.241535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f081-678f-4905-b95b-b9e5fbecddc8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.242196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f081-678f-4905-b95b-b9e5fbecddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.243155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f081-678f-4905-b95b-b9e5fbecddc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.291340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 862/1742] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:09.331725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-11a5f081-678f-4905-b95b-b9e5fbecddc8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:09.339706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-11a5f081-678f-4905-b95b-b9e5fbecddc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:09.353376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ad9c8-9530-4550-9edb-aea042fc271e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:09.354882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6b6ad9c8-9530-4550-9edb-aea042fc271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:09.361897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 881/1743] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:09.363144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5f081-678f-4905-b95b-b9e5fbecddc8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:09.409960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5f081-678f-4905-b95b-b9e5fbecddc8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:09.420242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:09.421111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.421401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.421681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.422457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:09.423020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:09.425464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:09.455199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 882/1744] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:09.457401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:09.483892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e6f6-a2de-44a3-9746-ccf221940973 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:09.498138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d78223-cdcb-4352-8af9-357ffb8aef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:09.499941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78223-cdcb-4352-8af9-357ffb8aef4d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.500979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78223-cdcb-4352-8af9-357ffb8aef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.502409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78223-cdcb-4352-8af9-357ffb8aef4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.504950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d78223-cdcb-4352-8af9-357ffb8aef4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:09.511168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d78223-cdcb-4352-8af9-357ffb8aef4d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:09.565447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ad9c8-9530-4550-9edb-aea042fc271e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:09.579067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26afc20-efc5-4cab-8c2d-887c8c60011e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:09.579922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afc20-efc5-4cab-8c2d-887c8c60011e service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.580617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afc20-efc5-4cab-8c2d-887c8c60011e service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.580617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afc20-efc5-4cab-8c2d-887c8c60011e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.604219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26afc20-efc5-4cab-8c2d-887c8c60011e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:09.604219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26afc20-efc5-4cab-8c2d-887c8c60011e service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:09.633149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 863/1745] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:09.649919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26afc20-efc5-4cab-8c2d-887c8c60011e service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:09.650640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 883/1746] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:09 2022] POST /identity/v3/projects => generated 363 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:09.665412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26afc20-efc5-4cab-8c2d-887c8c60011e service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:09.812223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489584e-192c-49fa-a827-e6822e4584b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:09.815176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489584e-192c-49fa-a827-e6822e4584b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.815466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489584e-192c-49fa-a827-e6822e4584b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.815748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489584e-192c-49fa-a827-e6822e4584b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.817572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489584e-192c-49fa-a827-e6822e4584b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:09.819812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489584e-192c-49fa-a827-e6822e4584b4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:09.834734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:09.886930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 864/1747] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:09 2022] POST /identity/v3/users => generated 282 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:09.890357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d78223-cdcb-4352-8af9-357ffb8aef4d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:09.905625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296827d0-16d1-41ad-acc9-06ae62070a4e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.909154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296827d0-16d1-41ad-acc9-06ae62070a4e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.909487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296827d0-16d1-41ad-acc9-06ae62070a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.916187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d78223-cdcb-4352-8af9-357ffb8aef4d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:09.926925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f39202-8205-407e-b6f8-f6190aad5ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:09.927552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f39202-8205-407e-b6f8-f6190aad5ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:09.927945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f39202-8205-407e-b6f8-f6190aad5ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:09.928218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f39202-8205-407e-b6f8-f6190aad5ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:09.928894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f39202-8205-407e-b6f8-f6190aad5ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:09.938259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f39202-8205-407e-b6f8-f6190aad5ad8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:09.970440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 884/1748] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:09.971392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f39202-8205-407e-b6f8-f6190aad5ad8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:09.997762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f39202-8205-407e-b6f8-f6190aad5ad8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:10.012705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08438f8-ed61-45ce-afa3-7083486f8ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:10.013341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08438f8-ed61-45ce-afa3-7083486f8ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:10.013636 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08438f8-ed61-45ce-afa3-7083486f8ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:10.013890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08438f8-ed61-45ce-afa3-7083486f8ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:10.014543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08438f8-ed61-45ce-afa3-7083486f8ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:10.017603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08438f8-ed61-45ce-afa3-7083486f8ff1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:10.028909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 885/1749] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:10.033962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08438f8-ed61-45ce-afa3-7083486f8ff1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:10.034826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-296827d0-16d1-41ad-acc9-06ae62070a4e None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:10.036326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-296827d0-16d1-41ad-acc9-06ae62070a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:10.061656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08438f8-ed61-45ce-afa3-7083486f8ff1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:10.075259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326515c-e832-4387-a2c8-890dcb71e3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:10.077089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326515c-e832-4387-a2c8-890dcb71e3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:10.078293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326515c-e832-4387-a2c8-890dcb71e3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:10.080944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326515c-e832-4387-a2c8-890dcb71e3a1 admin admin] PATH_INFO: `/v3/projects/c059a47bb80b4a0094221b54305eb3a9/users/e1623b5a51fe4dd8ab634d55258a7082/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:10.122322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326515c-e832-4387-a2c8-890dcb71e3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c059a47bb80b4a0094221b54305eb3a9, user_id=e1623b5a51fe4dd8ab634d55258a7082, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:10.127565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326515c-e832-4387-a2c8-890dcb71e3a1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:10.192238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 886/1750] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:10 2022] PUT /identity/v3/projects/c059a47bb80b4a0094221b54305eb3a9/users/e1623b5a51fe4dd8ab634d55258a7082/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:10.205727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cb908-b891-4f4c-b3b4-a521658daa1d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:10.206154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cb908-b891-4f4c-b3b4-a521658daa1d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:10.207415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cb908-b891-4f4c-b3b4-a521658daa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:10.277666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 865/1751] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:10.309922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9e2cb908-b891-4f4c-b3b4-a521658daa1d None None] MFA Rules not processed for user `e1623b5a51fe4dd8ab634d55258a7082`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:10.311366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9e2cb908-b891-4f4c-b3b4-a521658daa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:10.441519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 887/1752] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:10.446446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296827d0-16d1-41ad-acc9-06ae62070a4e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:10.463273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296827d0-16d1-41ad-acc9-06ae62070a4e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:10.553773 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cb908-b891-4f4c-b3b4-a521658daa1d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:10.557197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9e2cb908-b891-4f4c-b3b4-a521658daa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:10.584800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3384-a786-453d-9bef-a82b1f6fdd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:10.586503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3384-a786-453d-9bef-a82b1f6fdd34 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:10.587033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3384-a786-453d-9bef-a82b1f6fdd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:10.587381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3384-a786-453d-9bef-a82b1f6fdd34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:10.588100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec3384-a786-453d-9bef-a82b1f6fdd34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:10.590617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec3384-a786-453d-9bef-a82b1f6fdd34 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:10.640155 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 866/1753] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:10 2022] POST /identity/v3/projects => generated 363 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:10.641613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3384-a786-453d-9bef-a82b1f6fdd34 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:10.671646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:10.762485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cb908-b891-4f4c-b3b4-a521658daa1d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:10.791895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:10.833278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3384-a786-453d-9bef-a82b1f6fdd34 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:10.847968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:10.849546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:10.849830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:10.850107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:10.854208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:10.859044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:10.911021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 867/1754] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:10 2022] POST /identity/v3/users => generated 288 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:10.912253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:10.918726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:10.952568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c853557-2037-48ea-afb7-05458e12b6d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:10.953407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853557-2037-48ea-afb7-05458e12b6d3 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:10.953861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853557-2037-48ea-afb7-05458e12b6d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:10.954921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853557-2037-48ea-afb7-05458e12b6d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:10.957583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f05a28-27fa-4006-9603-8cb8c29edcc8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:10.973742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:10.975193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:10.975486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:10.977952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] PATH_INFO: `/v3/users/caf856d042ce4df5a0779c78e4da0658` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:10.979678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf856d042ce4df5a0779c78e4da0658)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:10.982476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:11.077979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf856d042ce4df5a0779c78e4da0658'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.103456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf856d042ce4df5a0779c78e4da0658'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.113411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf856d042ce4df5a0779c78e4da0658'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.119127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf856d042ce4df5a0779c78e4da0658'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.121342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf856d042ce4df5a0779c78e4da0658'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.145697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c853557-2037-48ea-afb7-05458e12b6d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:11.151638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c853557-2037-48ea-afb7-05458e12b6d3 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:11.169719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 868/1755] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:10 2022] DELETE /identity/v3/users/caf856d042ce4df5a0779c78e4da0658 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:11.175999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:11.177630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 888/1756] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:14:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:14:11.179028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda353dd-c474-44aa-91ab-b5c47d945ae9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:11.182970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda353dd-c474-44aa-91ab-b5c47d945ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:11.183330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda353dd-c474-44aa-91ab-b5c47d945ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:11.197149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:11.233168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cda353dd-c474-44aa-91ab-b5c47d945ae9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:11.238649 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cda353dd-c474-44aa-91ab-b5c47d945ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:11.523803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 889/1757] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:11.529504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda353dd-c474-44aa-91ab-b5c47d945ae9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:11.533282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cda353dd-c474-44aa-91ab-b5c47d945ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:11.551451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d1235-9bc2-4601-b190-c4a476d3a756 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:11.561277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808c2c5-5996-4ccf-a797-fc953cfe2dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:11.562048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808c2c5-5996-4ccf-a797-fc953cfe2dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:11.562238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808c2c5-5996-4ccf-a797-fc953cfe2dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:11.562520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808c2c5-5996-4ccf-a797-fc953cfe2dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:11.562702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda353dd-c474-44aa-91ab-b5c47d945ae9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:11.563310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808c2c5-5996-4ccf-a797-fc953cfe2dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:11.564939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808c2c5-5996-4ccf-a797-fc953cfe2dfe admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:11.576905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:11.578585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:11.579498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:11.581314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] PATH_INFO: `/v3/users/07f80bede751418c8ab5c26faba0bc95` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:11.585169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f80bede751418c8ab5c26faba0bc95)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:11.588302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:11.592773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 869/1758] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:11.597755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808c2c5-5996-4ccf-a797-fc953cfe2dfe admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:11.610856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:11.634166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808c2c5-5996-4ccf-a797-fc953cfe2dfe admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:11.648282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fe60c-df11-4f0d-809b-c5be33920bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:11.650019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fe60c-df11-4f0d-809b-c5be33920bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:11.650451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fe60c-df11-4f0d-809b-c5be33920bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:11.650918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fe60c-df11-4f0d-809b-c5be33920bb0 admin admin] PATH_INFO: `/v3/projects/6b8c678a00694cb796afb4079819caf8/users/4305f3c5b69d4e9cad902ba69df6726a/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:11.676765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fe60c-df11-4f0d-809b-c5be33920bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8c678a00694cb796afb4079819caf8, user_id=4305f3c5b69d4e9cad902ba69df6726a, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:11.682455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fe60c-df11-4f0d-809b-c5be33920bb0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:11.702785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 870/1759] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:11 2022] PUT /identity/v3/projects/6b8c678a00694cb796afb4079819caf8/users/4305f3c5b69d4e9cad902ba69df6726a/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:11.706968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fe60c-df11-4f0d-809b-c5be33920bb0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:11.784632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:11.797736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f80bede751418c8ab5c26faba0bc95'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.826017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f80bede751418c8ab5c26faba0bc95'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.844938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f80bede751418c8ab5c26faba0bc95'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.847029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f80bede751418c8ab5c26faba0bc95'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.849061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8024d141-5465-47b9-abd3-548d66092dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f80bede751418c8ab5c26faba0bc95'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:11.881322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fe60c-df11-4f0d-809b-c5be33920bb0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:11.896702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2b688-45c7-4936-9b1e-32077b1c9166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:11.897364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2b688-45c7-4936-9b1e-32077b1c9166 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:11.897850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2b688-45c7-4936-9b1e-32077b1c9166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:11.898829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2b688-45c7-4936-9b1e-32077b1c9166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:11.902071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2b688-45c7-4936-9b1e-32077b1c9166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:11.906726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2b688-45c7-4936-9b1e-32077b1c9166 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:11.915333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 890/1760] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:11 2022] DELETE /identity/v3/users/07f80bede751418c8ab5c26faba0bc95 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:11.915333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7bedd-2590-45cf-8f75-34d0948d8c52 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:11.915333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7bedd-2590-45cf-8f75-34d0948d8c52 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:11.915333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7bedd-2590-45cf-8f75-34d0948d8c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:11.942417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 871/1761] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:11.943989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50320a6-c1b1-4c6e-99a8-5a1bb216ee8a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:11.944285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50320a6-c1b1-4c6e-99a8-5a1bb216ee8a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:11.944590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50320a6-c1b1-4c6e-99a8-5a1bb216ee8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:11.986156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a7e7bedd-2590-45cf-8f75-34d0948d8c52 None None] MFA Rules not processed for user `31183ac0fdd642e6a1a7578899aa9fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:11.987620 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a7e7bedd-2590-45cf-8f75-34d0948d8c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:12.000010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f50320a6-c1b1-4c6e-99a8-5a1bb216ee8a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:12.010974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f50320a6-c1b1-4c6e-99a8-5a1bb216ee8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:12.156779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 891/1762] 10.222.0.60 () {58 vars in 1015 bytes} [Tue Aug 9 19:14:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:12.157724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7bedd-2590-45cf-8f75-34d0948d8c52 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:12.172720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a7e7bedd-2590-45cf-8f75-34d0948d8c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:12.182585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 872/1763] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:12.214339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7bedd-2590-45cf-8f75-34d0948d8c52 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:12.221220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:12.221893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:12.222168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:12.222428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] PATH_INFO: `/v3/projects/6b8c678a00694cb796afb4079819caf8/users/4305f3c5b69d4e9cad902ba69df6726a/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:12.224421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8c678a00694cb796afb4079819caf8, user_id=4305f3c5b69d4e9cad902ba69df6726a, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:12.227177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:12.237276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50320a6-c1b1-4c6e-99a8-5a1bb216ee8a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:12.241197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f50320a6-c1b1-4c6e-99a8-5a1bb216ee8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:12.253930 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 892/1764] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:12 2022] PUT /identity/v3/projects/6b8c678a00694cb796afb4079819caf8/users/4305f3c5b69d4e9cad902ba69df6726a/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:12.254943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:12.256635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:12.287894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50320a6-c1b1-4c6e-99a8-5a1bb216ee8a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:12.374412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a84f97-62d4-46fc-9de7-56f576456f62 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:12.386806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:12.387074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:12.387408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:12.387677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:12.456327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2116a5-e29c-445e-83c3-510162e0696b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:12.457254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2116a5-e29c-445e-83c3-510162e0696b service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:12.460333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2116a5-e29c-445e-83c3-510162e0696b service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:12.460333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2116a5-e29c-445e-83c3-510162e0696b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:12.471970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:12.496522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2116a5-e29c-445e-83c3-510162e0696b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:12.496522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2116a5-e29c-445e-83c3-510162e0696b service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:12.534815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 873/1765] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:12.542580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d0c8a-c1be-4491-88e4-3849a93c5851 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:12.542710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d0c8a-c1be-4491-88e4-3849a93c5851 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:12.542994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d0c8a-c1be-4491-88e4-3849a93c5851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:12.561525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:12.566542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:12.607980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 893/1766] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:12.608724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:12.615613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:12.658524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-650d0c8a-c1be-4491-88e4-3849a93c5851 None None] MFA Rules not processed for user `4305f3c5b69d4e9cad902ba69df6726a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:12.660464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-650d0c8a-c1be-4491-88e4-3849a93c5851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:12.662284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a2835-3c38-43f8-b5ec-a7c7cefa56dc service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:12.674617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:12.678286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:12.679150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:12.680493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:12.686961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:12.731500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:12.733282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:12.747873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 894/1767] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:12.824334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:12.902578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 874/1768] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:12.911191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:12.912906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:13.021868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d0c8a-c1be-4491-88e4-3849a93c5851 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:13.024102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-650d0c8a-c1be-4491-88e4-3849a93c5851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:13.064776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d0c8a-c1be-4491-88e4-3849a93c5851 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:13.073914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:13.075509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:13.076308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:13.077082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] PATH_INFO: `/v3/projects/50f6078d20984fd5adb321995df36de1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:13.079079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=50f6078d20984fd5adb321995df36de1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:13.079821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256962b-0bf8-4f7e-91f2-a0e6b56de25f service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:13.086992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:13.089597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d906e2-dab4-4cb4-a191-72af25e179ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:13.090738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d906e2-dab4-4cb4-a191-72af25e179ac service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:13.091785 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d906e2-dab4-4cb4-a191-72af25e179ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:13.093101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d906e2-dab4-4cb4-a191-72af25e179ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:13.116305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d906e2-dab4-4cb4-a191-72af25e179ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:13.119816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d906e2-dab4-4cb4-a191-72af25e179ac service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:13.142596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 895/1769] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:13.293290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d906e2-dab4-4cb4-a191-72af25e179ac service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:13.307929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d906e2-dab4-4cb4-a191-72af25e179ac service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:13.317862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50f6078d20984fd5adb321995df36de1', 'request_id': 'req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:13.350975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50f6078d20984fd5adb321995df36de1', 'request_id': 'req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:13.402347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50f6078d20984fd5adb321995df36de1', 'request_id': 'req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:13.448483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] The token cache is being invalidate because project 50f6078d20984fd5adb321995df36de1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:13.450743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:13.453482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:13.458143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 875/1770] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:13 2022] DELETE /identity/v3/projects/50f6078d20984fd5adb321995df36de1 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:13.464150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:13.470701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:13.492848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:13.493376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:13.493658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:13.494011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:13.500402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:13.701800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:13.703673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:13.743148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 896/1771] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:13.787057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62abd0-bd5f-4e10-bd2d-b8e54a2fb2da admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:13.797314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:13.798803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:13.799511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:13.800286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] PATH_INFO: `/v3/users/ee9221c2d127455fad94498069b05ede` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:13.803192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9221c2d127455fad94498069b05ede)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:13.807682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:14.003130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9221c2d127455fad94498069b05ede'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:14.036469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9221c2d127455fad94498069b05ede'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:14.071357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9221c2d127455fad94498069b05ede'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:14.072340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9221c2d127455fad94498069b05ede'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:14.076387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9221c2d127455fad94498069b05ede'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:14.112795 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:14.159645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 876/1772] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:13 2022] DELETE /identity/v3/users/ee9221c2d127455fad94498069b05ede => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:14.915537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:14.917767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:14.941290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44944b04-6fa2-4a33-a024-118ceb1dadc5 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:14.962596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:15.071097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:15.071756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:15.072030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:15.072299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] PATH_INFO: `/v3/projects/eae32c33b0744829bb453166d0240d8e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:15.073644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae32c33b0744829bb453166d0240d8e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:15.075598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:15.206250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae32c33b0744829bb453166d0240d8e', 'request_id': 'req-2ed2af1f-2a9d-4859-b947-005a29d4fd39'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.216804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae32c33b0744829bb453166d0240d8e', 'request_id': 'req-2ed2af1f-2a9d-4859-b947-005a29d4fd39'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.242979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae32c33b0744829bb453166d0240d8e', 'request_id': 'req-2ed2af1f-2a9d-4859-b947-005a29d4fd39'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.306426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] The token cache is being invalidate because project eae32c33b0744829bb453166d0240d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:15.306733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.309427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.312188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 897/1773] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:14 2022] DELETE /identity/v3/projects/eae32c33b0744829bb453166d0240d8e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:15.316419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:15.318131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:15.366666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a60404-dc28-4221-b6ac-3edb647805fc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:15.531692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:15.533474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:15.533961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:15.534257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] PATH_INFO: `/v3/users/1122c364ab1141a7a74bca56efc94892` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:15.535719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1122c364ab1141a7a74bca56efc94892)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:15.540003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:15.719201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1122c364ab1141a7a74bca56efc94892'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.739754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1122c364ab1141a7a74bca56efc94892'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.754630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1122c364ab1141a7a74bca56efc94892'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.760459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1122c364ab1141a7a74bca56efc94892'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.761677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5e418016-7081-4b02-9e5c-697f3cee3e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1122c364ab1141a7a74bca56efc94892'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:15.822126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 877/1774] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:15 2022] DELETE /identity/v3/users/1122c364ab1141a7a74bca56efc94892 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:16.164328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:16.166687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:16.228088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2af1f-2a9d-4859-b947-005a29d4fd39 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:16.390351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b775-4e25-411b-ad66-327a05125ec6 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:16.390913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b775-4e25-411b-ad66-327a05125ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:16.391019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b775-4e25-411b-ad66-327a05125ec6 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:16.393111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 878/1775] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:14:16.411675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc802-f916-4cce-afe9-8edbf70ec506 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:16.412652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc802-f916-4cce-afe9-8edbf70ec506 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:16.413128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc802-f916-4cce-afe9-8edbf70ec506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:16.441607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:16.444273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:16.444273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:16.444273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] PATH_INFO: `/v3/projects/73dba1bf84484e10b1fc5f7135b08985` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:16.446534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73dba1bf84484e10b1fc5f7135b08985)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:16.449739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:16.489495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e64cc802-f916-4cce-afe9-8edbf70ec506 None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:16.497114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e64cc802-f916-4cce-afe9-8edbf70ec506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:16.647689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:16.652271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73dba1bf84484e10b1fc5f7135b08985', 'request_id': 'req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:16.680060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73dba1bf84484e10b1fc5f7135b08985', 'request_id': 'req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:16.696036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73dba1bf84484e10b1fc5f7135b08985', 'request_id': 'req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:16.726946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] The token cache is being invalidate because project 73dba1bf84484e10b1fc5f7135b08985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:16.727395 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:16.731686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:16.734462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 898/1776] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:16 2022] DELETE /identity/v3/projects/73dba1bf84484e10b1fc5f7135b08985 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:16.737857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 879/1777] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:14:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:16.738965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:16.742914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:16.779401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc802-f916-4cce-afe9-8edbf70ec506 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:16.781462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e64cc802-f916-4cce-afe9-8edbf70ec506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:16.797972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:16.812778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc802-f916-4cce-afe9-8edbf70ec506 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:16.918644 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:16.919311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:16.919626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:16.919857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] PATH_INFO: `/v3/projects/8c20c9a46a9244a6b51e4bae06052d85` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:16.922508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff8a3b-fba6-46c8-b45c-c83b9a7b7609 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:16.934269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c20c9a46a9244a6b51e4bae06052d85)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:16.937900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:16.938194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:16.938500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:16.938841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:16.939033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] PATH_INFO: `/v3/users/4e0dacd71b784c26a0a30f5991534ef4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:16.955923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0dacd71b784c26a0a30f5991534ef4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:16.963168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:17.056673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c20c9a46a9244a6b51e4bae06052d85', 'request_id': 'req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.068397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c20c9a46a9244a6b51e4bae06052d85', 'request_id': 'req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.084038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c20c9a46a9244a6b51e4bae06052d85', 'request_id': 'req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.137968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] The token cache is being invalidate because project 8c20c9a46a9244a6b51e4bae06052d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:17.138515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.143973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.146071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 880/1778] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:16 2022] DELETE /identity/v3/projects/8c20c9a46a9244a6b51e4bae06052d85 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:17.146968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:17.151002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:17.170193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0dacd71b784c26a0a30f5991534ef4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.189726 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0dacd71b784c26a0a30f5991534ef4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.214359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0dacd71b784c26a0a30f5991534ef4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.215160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0dacd71b784c26a0a30f5991534ef4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.215837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c7c2478-cbfc-4373-953c-d063b8d9c58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0dacd71b784c26a0a30f5991534ef4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:17.257524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 899/1779] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:16 2022] DELETE /identity/v3/users/4e0dacd71b784c26a0a30f5991534ef4 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:17.262444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda96-b3ea-4e3f-85aa-b3069e051408 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:17.263039 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda96-b3ea-4e3f-85aa-b3069e051408 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:17.263487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda96-b3ea-4e3f-85aa-b3069e051408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:17.307477 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-534eda96-b3ea-4e3f-85aa-b3069e051408 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:17.308372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-534eda96-b3ea-4e3f-85aa-b3069e051408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:17.366617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84b900-cdeb-4581-9bb3-eaeab7d7b13c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:17.453479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:17.505288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 900/1780] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:17.506438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18122e48-e545-4a18-97bd-36d5ec7cf2b3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:17.507222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18122e48-e545-4a18-97bd-36d5ec7cf2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:17.507222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18122e48-e545-4a18-97bd-36d5ec7cf2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:17.521868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:17.564109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-18122e48-e545-4a18-97bd-36d5ec7cf2b3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:17.565386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-18122e48-e545-4a18-97bd-36d5ec7cf2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:17.615020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 901/1781] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:17.616023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18122e48-e545-4a18-97bd-36d5ec7cf2b3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:17.649259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:17.650408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:17.650794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:17.651011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:17.657604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:17.661140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18122e48-e545-4a18-97bd-36d5ec7cf2b3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:17.667946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:17.668622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:17.668904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:17.669174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:17.670181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:17.670301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:17.671923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:17.682037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:17.683646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:17.708806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 902/1782] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:17.709757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:17.718089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:17.742481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf27671-32ef-4ae6-a339-4b1e0902554c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:17.751741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 881/1783] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:17.752869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:17.767808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:17.769029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:17.769447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:17.769869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] PATH_INFO: `/v3/users/e1623b5a51fe4dd8ab634d55258a7082` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:17.771765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1623b5a51fe4dd8ab634d55258a7082)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:17.774330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8958ec-58a5-40bd-9714-81adfb5a4bd2 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:17.775332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:17.784659 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec335029-b907-4bc3-97d6-00c797acfa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:17.785256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335029-b907-4bc3-97d6-00c797acfa9f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:17.785567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335029-b907-4bc3-97d6-00c797acfa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:17.785842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec335029-b907-4bc3-97d6-00c797acfa9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:17.787910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec335029-b907-4bc3-97d6-00c797acfa9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:17.794412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec335029-b907-4bc3-97d6-00c797acfa9f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:17.909784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 882/1784] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:17 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:17.919343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec335029-b907-4bc3-97d6-00c797acfa9f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:17.955942 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1623b5a51fe4dd8ab634d55258a7082'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.004228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1623b5a51fe4dd8ab634d55258a7082'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.030073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1623b5a51fe4dd8ab634d55258a7082'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.031768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1623b5a51fe4dd8ab634d55258a7082'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.036594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-871dc075-a58d-4ed9-8d34-af4685a10229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1623b5a51fe4dd8ab634d55258a7082'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.096783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 903/1785] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:17 2022] DELETE /identity/v3/users/e1623b5a51fe4dd8ab634d55258a7082 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:18.098271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062d4d6-6bc5-45ad-a484-46ced78173a6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:18.098536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062d4d6-6bc5-45ad-a484-46ced78173a6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:18.098815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062d4d6-6bc5-45ad-a484-46ced78173a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:18.131741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a062d4d6-6bc5-45ad-a484-46ced78173a6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:18.133770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a062d4d6-6bc5-45ad-a484-46ced78173a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:18.166187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec335029-b907-4bc3-97d6-00c797acfa9f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:18.311446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 904/1786] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:18.328723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:18.329408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:18.329689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:18.329966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:18.332004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:18.334022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:18.378527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 883/1787] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:17 2022] POST /identity/v3/users => generated 287 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:18.388219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062d4d6-6bc5-45ad-a484-46ced78173a6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:18.391088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a062d4d6-6bc5-45ad-a484-46ced78173a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:18.425899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:18.428086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:18.456890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062d4d6-6bc5-45ad-a484-46ced78173a6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:18.469166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:18.471288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:18.472175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:18.472902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:18.473495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1333f05-1c1b-4679-92d7-6dda5e4261e8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:18.473615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:18.477834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:18.492673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:18.493615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:18.493988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:18.494393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] PATH_INFO: `/v3/projects/15e0d0eb463a4bbc83ffdca351bbf762` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:18.498141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e0d0eb463a4bbc83ffdca351bbf762)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:18.501051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:18.516722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 905/1788] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:18.518028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:18.520729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:18.622517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:18.664506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:18.667053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e0d0eb463a4bbc83ffdca351bbf762', 'request_id': 'req-39413ee7-bfbd-4314-bc18-e6e71d7d3118'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.688066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e0d0eb463a4bbc83ffdca351bbf762', 'request_id': 'req-39413ee7-bfbd-4314-bc18-e6e71d7d3118'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.704979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e0d0eb463a4bbc83ffdca351bbf762', 'request_id': 'req-39413ee7-bfbd-4314-bc18-e6e71d7d3118'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.738640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42b65b-e35c-4b4f-8193-557ea97759f7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:18.753384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] The token cache is being invalidate because project 15e0d0eb463a4bbc83ffdca351bbf762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:18.753688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.755065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:18.763904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 884/1789] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:18 2022] DELETE /identity/v3/projects/15e0d0eb463a4bbc83ffdca351bbf762 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:18.765632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:18.771270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:18.950770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57f4ed-97e0-4305-af62-ba6208dba594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:18.951802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f4ed-97e0-4305-af62-ba6208dba594 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:18.953634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f4ed-97e0-4305-af62-ba6208dba594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:18.955005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f4ed-97e0-4305-af62-ba6208dba594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:18.959855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3b57f4ed-97e0-4305-af62-ba6208dba594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:19.005774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39413ee7-bfbd-4314-bc18-e6e71d7d3118 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:19.014740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25858bb5-6e99-4f26-88db-4b9817fd079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:19.015383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858bb5-6e99-4f26-88db-4b9817fd079f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.015668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858bb5-6e99-4f26-88db-4b9817fd079f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.015944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858bb5-6e99-4f26-88db-4b9817fd079f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.016641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858bb5-6e99-4f26-88db-4b9817fd079f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:19.018928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858bb5-6e99-4f26-88db-4b9817fd079f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:19.039120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 885/1790] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:19.046719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753e8d9-a598-480a-9626-1797461bf4f2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.047195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753e8d9-a598-480a-9626-1797461bf4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.047567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753e8d9-a598-480a-9626-1797461bf4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.103741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6753e8d9-a598-480a-9626-1797461bf4f2 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:19.108132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6753e8d9-a598-480a-9626-1797461bf4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:19.137205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 886/1791] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:19.137818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753e8d9-a598-480a-9626-1797461bf4f2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:19.154406 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753e8d9-a598-480a-9626-1797461bf4f2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:19.169784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece112c-98d1-462a-9f69-fd2c8614dcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:19.177432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece112c-98d1-462a-9f69-fd2c8614dcef admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.177750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece112c-98d1-462a-9f69-fd2c8614dcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.178056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece112c-98d1-462a-9f69-fd2c8614dcef admin admin] PATH_INFO: `/v3/projects/9cf283d0173e4c6f936c5b3a8ce18113/users/748477083e514f32ac1d40f0fa84762b/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.223194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57f4ed-97e0-4305-af62-ba6208dba594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:19.224970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece112c-98d1-462a-9f69-fd2c8614dcef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf283d0173e4c6f936c5b3a8ce18113, user_id=748477083e514f32ac1d40f0fa84762b, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:19.225204 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57f4ed-97e0-4305-af62-ba6208dba594 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:19.235828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece112c-98d1-462a-9f69-fd2c8614dcef admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:19.264087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 906/1792] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 747 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:19.265960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae0146-22d1-426e-8e20-5598a61c4e04 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.266246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae0146-22d1-426e-8e20-5598a61c4e04 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.266681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae0146-22d1-426e-8e20-5598a61c4e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.270454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 887/1793] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:19 2022] PUT /identity/v3/projects/9cf283d0173e4c6f936c5b3a8ce18113/users/748477083e514f32ac1d40f0fa84762b/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:19.295092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3bae0146-22d1-426e-8e20-5598a61c4e04 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:19.295713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b6d41-99b9-412b-831a-0ddebe45becb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.296003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b6d41-99b9-412b-831a-0ddebe45becb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.296279 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b6d41-99b9-412b-831a-0ddebe45becb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.296565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3bae0146-22d1-426e-8e20-5598a61c4e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:19.381148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-645b6d41-99b9-412b-831a-0ddebe45becb None None] MFA Rules not processed for user `748477083e514f32ac1d40f0fa84762b`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:19.388004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-645b6d41-99b9-412b-831a-0ddebe45becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:19.485850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 907/1794] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:19.489855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0047-53bd-4575-a6b4-781167b5d3ab None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.490549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0047-53bd-4575-a6b4-781167b5d3ab None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.492262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0047-53bd-4575-a6b4-781167b5d3ab None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.498510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 908/1795] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:14:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:14:19.500385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e0047-53bd-4575-a6b4-781167b5d3ab None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:19.537897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e0047-53bd-4575-a6b4-781167b5d3ab None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:19.552346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d7d3c-643b-4ea4-aec5-1cfe1cf8d47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:19.556175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7d3c-643b-4ea4-aec5-1cfe1cf8d47e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.558648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7d3c-643b-4ea4-aec5-1cfe1cf8d47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.559729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7d3c-643b-4ea4-aec5-1cfe1cf8d47e admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.560647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d7d3c-643b-4ea4-aec5-1cfe1cf8d47e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:19.561105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d7d3c-643b-4ea4-aec5-1cfe1cf8d47e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:19.563624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d7d3c-643b-4ea4-aec5-1cfe1cf8d47e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:19.586233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 909/1796] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:19.588720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4fa8e-27e7-4dcd-9f35-46785a1081b8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.589234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4fa8e-27e7-4dcd-9f35-46785a1081b8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.589705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4fa8e-27e7-4dcd-9f35-46785a1081b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.602415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 888/1797] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:19.616173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b6d41-99b9-412b-831a-0ddebe45becb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:19.650980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b6d41-99b9-412b-831a-0ddebe45becb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:19.654204 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d1b4fa8e-27e7-4dcd-9f35-46785a1081b8 None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:19.657328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d1b4fa8e-27e7-4dcd-9f35-46785a1081b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:19.667127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:19.669197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.671116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.671249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:19.672516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:19.677767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:19.695752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:19.743981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:19.750039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 889/1798] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:19 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:19.751135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:19.755082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:19.853597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 910/1799] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:14:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:19.854821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4fa8e-27e7-4dcd-9f35-46785a1081b8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:19.873959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4fa8e-27e7-4dcd-9f35-46785a1081b8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:19.922691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b63066-5f6c-41dd-83f4-e9871c0d1729 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:19.940248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d29c9-1ba1-4729-9d99-abb1e7481985 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:19.941113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d29c9-1ba1-4729-9d99-abb1e7481985 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:19.941564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d29c9-1ba1-4729-9d99-abb1e7481985 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:19.942152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d29c9-1ba1-4729-9d99-abb1e7481985 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:20.082791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111701f2-3369-4d0c-9c39-1cf3065a24d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:20.083800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111701f2-3369-4d0c-9c39-1cf3065a24d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:20.089280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111701f2-3369-4d0c-9c39-1cf3065a24d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:20.089890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111701f2-3369-4d0c-9c39-1cf3065a24d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:20.090821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111701f2-3369-4d0c-9c39-1cf3065a24d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:20.093708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111701f2-3369-4d0c-9c39-1cf3065a24d1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:20.129016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d29c9-1ba1-4729-9d99-abb1e7481985 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:20.131768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d29c9-1ba1-4729-9d99-abb1e7481985 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:20.134628 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 911/1800] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:19 2022] POST /identity/v3/users => generated 290 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:20.140286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111701f2-3369-4d0c-9c39-1cf3065a24d1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:20.146255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 890/1801] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:20.171466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111701f2-3369-4d0c-9c39-1cf3065a24d1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:20.178720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ab397-826e-4bb3-aa1c-0b86df9ff2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:20.180322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ab397-826e-4bb3-aa1c-0b86df9ff2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:20.180713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ab397-826e-4bb3-aa1c-0b86df9ff2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:20.181072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ab397-826e-4bb3-aa1c-0b86df9ff2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:20.181781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ab397-826e-4bb3-aa1c-0b86df9ff2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:20.184704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ab397-826e-4bb3-aa1c-0b86df9ff2b5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:20.239394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 912/1802] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:20.240545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d29c9-1ba1-4729-9d99-abb1e7481985 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:20.245001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ab397-826e-4bb3-aa1c-0b86df9ff2b5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:20.268744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d29c9-1ba1-4729-9d99-abb1e7481985 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:20.273346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ab397-826e-4bb3-aa1c-0b86df9ff2b5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:20.301794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3893fa-50c6-49e3-8bed-04c768032a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:20.305070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9099a-e840-4e55-a3e0-5da09dbe7f16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:20.306525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3893fa-50c6-49e3-8bed-04c768032a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:20.306963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3893fa-50c6-49e3-8bed-04c768032a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:20.307748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9099a-e840-4e55-a3e0-5da09dbe7f16 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:20.308201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3893fa-50c6-49e3-8bed-04c768032a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:20.313194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9099a-e840-4e55-a3e0-5da09dbe7f16 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:20.313474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9099a-e840-4e55-a3e0-5da09dbe7f16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:20.314438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3893fa-50c6-49e3-8bed-04c768032a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:20.316082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3893fa-50c6-49e3-8bed-04c768032a41 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:20.364905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9099a-e840-4e55-a3e0-5da09dbe7f16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:20.373247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9099a-e840-4e55-a3e0-5da09dbe7f16 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:20.407678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 891/1803] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:20.418816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9099a-e840-4e55-a3e0-5da09dbe7f16 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:20.419686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 913/1804] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:20.438672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9099a-e840-4e55-a3e0-5da09dbe7f16 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:20.453587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a7cf8-29dd-47e5-9e78-091bbef44814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:20.455782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a7cf8-29dd-47e5-9e78-091bbef44814 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:20.456086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a7cf8-29dd-47e5-9e78-091bbef44814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:20.456333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a7cf8-29dd-47e5-9e78-091bbef44814 admin admin] PATH_INFO: `/v3/projects/c224332f55774723b08996c61bf3186a/users/4a0dd1194187418aa0fd5ee55e7f4075/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:20.507100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a7cf8-29dd-47e5-9e78-091bbef44814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c224332f55774723b08996c61bf3186a, user_id=4a0dd1194187418aa0fd5ee55e7f4075, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:20.510551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a7cf8-29dd-47e5-9e78-091bbef44814 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:20.570292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 892/1805] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:20 2022] PUT /identity/v3/projects/c224332f55774723b08996c61bf3186a/users/4a0dd1194187418aa0fd5ee55e7f4075/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:20.580775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc2e09-6b43-442b-82a3-97bc48d7e68e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:20.581270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc2e09-6b43-442b-82a3-97bc48d7e68e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:20.581270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc2e09-6b43-442b-82a3-97bc48d7e68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:20.642353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-31dc2e09-6b43-442b-82a3-97bc48d7e68e None None] MFA Rules not processed for user `4a0dd1194187418aa0fd5ee55e7f4075`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:20.643555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-31dc2e09-6b43-442b-82a3-97bc48d7e68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:20.783264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 914/1806] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:20.787697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a7cf8-29dd-47e5-9e78-091bbef44814 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:20.862952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc2e09-6b43-442b-82a3-97bc48d7e68e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:20.865298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-31dc2e09-6b43-442b-82a3-97bc48d7e68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:20.912693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:20.916915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:20.982745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a7cf8-29dd-47e5-9e78-091bbef44814 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:21.007302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e57a1-7340-4373-b3c9-11a5c6ed6c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:21.012275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e57a1-7340-4373-b3c9-11a5c6ed6c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.013967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e57a1-7340-4373-b3c9-11a5c6ed6c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.014257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e57a1-7340-4373-b3c9-11a5c6ed6c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.017677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e57a1-7340-4373-b3c9-11a5c6ed6c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:21.040096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e57a1-7340-4373-b3c9-11a5c6ed6c2f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:21.060167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc2e09-6b43-442b-82a3-97bc48d7e68e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:21.072301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:21.085316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.085978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.088067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] PATH_INFO: `/v3/projects/52afcbdaf190427cb4b3cae73bfbdbef` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.094867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52afcbdaf190427cb4b3cae73bfbdbef)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:21.103072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:21.147577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 893/1807] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:20 2022] POST /identity/v3/projects => generated 379 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:21.147794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e57a1-7340-4373-b3c9-11a5c6ed6c2f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:21.248203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52afcbdaf190427cb4b3cae73bfbdbef', 'request_id': 'req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.263643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52afcbdaf190427cb4b3cae73bfbdbef', 'request_id': 'req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.292718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52afcbdaf190427cb4b3cae73bfbdbef', 'request_id': 'req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.351149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e57a1-7340-4373-b3c9-11a5c6ed6c2f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:21.351410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] The token cache is being invalidate because project 52afcbdaf190427cb4b3cae73bfbdbef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:21.351410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.356879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f66d46ed-a45e-41c5-91bd-03e8ba7754d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.364045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 915/1808] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:20 2022] DELETE /identity/v3/projects/52afcbdaf190427cb4b3cae73bfbdbef => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:21.371369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772b378-912f-451e-b205-0022f6f2b7cb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.371717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772b378-912f-451e-b205-0022f6f2b7cb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.371999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772b378-912f-451e-b205-0022f6f2b7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.447846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9772b378-912f-451e-b205-0022f6f2b7cb None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:21.463147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9772b378-912f-451e-b205-0022f6f2b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:21.511274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:21.512816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.513462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.514681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.516832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:21.519653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:21.590596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 894/1809] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:21 2022] POST /identity/v3/users => generated 296 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:21.593392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:21.602130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:21.622758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24e50-91e2-4686-a0f8-631215f23a1b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:21.642230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:21.642933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.643220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.643555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] PATH_INFO: `/v3/projects/c059a47bb80b4a0094221b54305eb3a9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.644763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 916/1810] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:21.645226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c059a47bb80b4a0094221b54305eb3a9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:21.645720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772b378-912f-451e-b205-0022f6f2b7cb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:21.647101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9772b378-912f-451e-b205-0022f6f2b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:21.649240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:21.682220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772b378-912f-451e-b205-0022f6f2b7cb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:21.704022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc26749-1c71-4abb-abf4-602c80930298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:21.704688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc26749-1c71-4abb-abf4-602c80930298 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.704969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc26749-1c71-4abb-abf4-602c80930298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.705517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc26749-1c71-4abb-abf4-602c80930298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.705897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc26749-1c71-4abb-abf4-602c80930298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:21.707948 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc26749-1c71-4abb-abf4-602c80930298 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:21.733730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 917/1811] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:21.737255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115883d0-eb7a-4ca4-a12b-f9363ff9c97b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.737255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115883d0-eb7a-4ca4-a12b-f9363ff9c97b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.737255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115883d0-eb7a-4ca4-a12b-f9363ff9c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.745779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c059a47bb80b4a0094221b54305eb3a9', 'request_id': 'req-2989e382-ec0d-4171-b401-11098d3d56cc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.753423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c059a47bb80b4a0094221b54305eb3a9', 'request_id': 'req-2989e382-ec0d-4171-b401-11098d3d56cc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.757128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-115883d0-eb7a-4ca4-a12b-f9363ff9c97b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:21.759151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-115883d0-eb7a-4ca4-a12b-f9363ff9c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:21.765349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:21.770990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c059a47bb80b4a0094221b54305eb3a9', 'request_id': 'req-2989e382-ec0d-4171-b401-11098d3d56cc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.795698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] The token cache is being invalidate because project c059a47bb80b4a0094221b54305eb3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:21.796349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.796785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.798868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 895/1812] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:21 2022] DELETE /identity/v3/projects/c059a47bb80b4a0094221b54305eb3a9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:21.800330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:21.801861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:21.850618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:21.892776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 918/1813] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:21.893764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115883d0-eb7a-4ca4-a12b-f9363ff9c97b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:21.894975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-115883d0-eb7a-4ca4-a12b-f9363ff9c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:21.899900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e382-ec0d-4171-b401-11098d3d56cc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:21.904747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:21.905538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.905892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.906158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] PATH_INFO: `/v3/projects/1064052adad347dd92c493818334bdfd/users/1819c9ce24f94b14af8b03ffcfcf343b/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.908481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115883d0-eb7a-4ca4-a12b-f9363ff9c97b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:21.913130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:21.914004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:21.914156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:21.914523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] PATH_INFO: `/v3/users/4305f3c5b69d4e9cad902ba69df6726a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:21.916147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4305f3c5b69d4e9cad902ba69df6726a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:21.917943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:21.922528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1064052adad347dd92c493818334bdfd, user_id=1819c9ce24f94b14af8b03ffcfcf343b, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:21.925465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:21.940134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 896/1814] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:21 2022] PUT /identity/v3/projects/1064052adad347dd92c493818334bdfd/users/1819c9ce24f94b14af8b03ffcfcf343b/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:21.941106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:21.942640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:21.971159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4305f3c5b69d4e9cad902ba69df6726a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.979921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4305f3c5b69d4e9cad902ba69df6726a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.988706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4305f3c5b69d4e9cad902ba69df6726a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.989344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4305f3c5b69d4e9cad902ba69df6726a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:21.990057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4305f3c5b69d4e9cad902ba69df6726a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.010101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 919/1815] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:21 2022] DELETE /identity/v3/users/4305f3c5b69d4e9cad902ba69df6726a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:22.010395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9fac2-af72-4ab2-977c-ef394a8b6cc4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.010922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.012373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.087124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.087840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.088125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.088421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] PATH_INFO: `/v3/users/748477083e514f32ac1d40f0fa84762b` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.089620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=748477083e514f32ac1d40f0fa84762b)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.091800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.093031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fadfe-234a-4644-b019-410f79557fae admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.099175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.099812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.100142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.100349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.101106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:22.101483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.103794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.116951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:22.120577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 920/1816] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:22.121518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.123385 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.137844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d38b2-013b-4149-82cd-6827f56dac1b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.144034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebf464-1289-4d23-91b5-772494ec2bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.144646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf464-1289-4d23-91b5-772494ec2bab admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.144998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf464-1289-4d23-91b5-772494ec2bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.145343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebf464-1289-4d23-91b5-772494ec2bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.146064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebf464-1289-4d23-91b5-772494ec2bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.147766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebf464-1289-4d23-91b5-772494ec2bab admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.150083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748477083e514f32ac1d40f0fa84762b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.160023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748477083e514f32ac1d40f0fa84762b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.161417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 921/1817] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:22.162081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebf464-1289-4d23-91b5-772494ec2bab admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.172854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebf464-1289-4d23-91b5-772494ec2bab admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.174120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748477083e514f32ac1d40f0fa84762b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.175220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748477083e514f32ac1d40f0fa84762b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.175902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748477083e514f32ac1d40f0fa84762b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.178422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c481038-ce9a-420a-8f38-382821d89164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.179814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c481038-ce9a-420a-8f38-382821d89164 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.180205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c481038-ce9a-420a-8f38-382821d89164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.180664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c481038-ce9a-420a-8f38-382821d89164 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.181380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c481038-ce9a-420a-8f38-382821d89164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.183405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c481038-ce9a-420a-8f38-382821d89164 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.214253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 897/1818] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:21 2022] DELETE /identity/v3/users/748477083e514f32ac1d40f0fa84762b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:22.215147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.216544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.219823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 922/1819] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:22 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:22.221240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01f54c-db34-4925-b8a9-4d2349206761 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.221713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01f54c-db34-4925-b8a9-4d2349206761 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.221868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01f54c-db34-4925-b8a9-4d2349206761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.241933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2a01f54c-db34-4925-b8a9-4d2349206761 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:22.243322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2a01f54c-db34-4925-b8a9-4d2349206761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.249080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:22.299225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e522f-ba8f-464f-ac89-a838d7418369 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.324596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 923/1820] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:22.325648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01f54c-db34-4925-b8a9-4d2349206761 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.333234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2a01f54c-db34-4925-b8a9-4d2349206761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.348161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01f54c-db34-4925-b8a9-4d2349206761 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.359183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.360078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.360522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.361126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.362502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.365228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.388566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.389312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.389793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.390075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] PATH_INFO: `/v3/projects/1064052adad347dd92c493818334bdfd/users/1819c9ce24f94b14af8b03ffcfcf343b/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.392702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1064052adad347dd92c493818334bdfd, user_id=1819c9ce24f94b14af8b03ffcfcf343b, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.399517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.401480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 924/1821] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:22 2022] POST /identity/v3/users => generated 284 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:22.402962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.405644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.414635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 898/1822] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:22 2022] PUT /identity/v3/projects/1064052adad347dd92c493818334bdfd/users/1819c9ce24f94b14af8b03ffcfcf343b/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:22.415359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.417627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.467895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:22.484216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693636-9023-4101-8a75-aa75e2d0cb47 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.492782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b8335-4182-4e1e-8d7d-e2a16c497d24 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.497877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.498633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.499043 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.499424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] PATH_INFO: `/v3/projects/6b8c678a00694cb796afb4079819caf8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.500690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8c678a00694cb796afb4079819caf8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.502705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.546515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0be1c-c17b-4253-a1f5-0a1f6918397f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.547146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0be1c-c17b-4253-a1f5-0a1f6918397f service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.547426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0be1c-c17b-4253-a1f5-0a1f6918397f service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.547725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0be1c-c17b-4253-a1f5-0a1f6918397f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.558390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0be1c-c17b-4253-a1f5-0a1f6918397f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.559837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0be1c-c17b-4253-a1f5-0a1f6918397f service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.569411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 925/1823] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:22.571258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0be1c-c17b-4253-a1f5-0a1f6918397f service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.574512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8c678a00694cb796afb4079819caf8', 'request_id': 'req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.581291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0be1c-c17b-4253-a1f5-0a1f6918397f service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.590610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8c678a00694cb796afb4079819caf8', 'request_id': 'req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.604229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8c678a00694cb796afb4079819caf8', 'request_id': 'req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.627861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] The token cache is being invalidate because project 6b8c678a00694cb796afb4079819caf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:22.628164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.628897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fe34ebe-a2cc-45e1-bf52-10ac5d06d840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:22.630714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 899/1824] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:22 2022] DELETE /identity/v3/projects/6b8c678a00694cb796afb4079819caf8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:22.631846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868b01e-f4dd-43d5-8f35-8ad5f61f7fce None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.632187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868b01e-f4dd-43d5-8f35-8ad5f61f7fce None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.633162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868b01e-f4dd-43d5-8f35-8ad5f61f7fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.653312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:22.674562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dc58a-7a67-4982-a1b1-6d49e4a22077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.675960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dc58a-7a67-4982-a1b1-6d49e4a22077 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.676090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dc58a-7a67-4982-a1b1-6d49e4a22077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.676430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dc58a-7a67-4982-a1b1-6d49e4a22077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.677572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498dc58a-7a67-4982-a1b1-6d49e4a22077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.679810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498dc58a-7a67-4982-a1b1-6d49e4a22077 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.682010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f868b01e-f4dd-43d5-8f35-8ad5f61f7fce None None] MFA Rules not processed for user `1819c9ce24f94b14af8b03ffcfcf343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:22.684133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f868b01e-f4dd-43d5-8f35-8ad5f61f7fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.691723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 926/1825] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:22.696424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19e0be-46f6-4d67-96af-d77d3f693623 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.696822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19e0be-46f6-4d67-96af-d77d3f693623 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.697172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19e0be-46f6-4d67-96af-d77d3f693623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.722859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6b19e0be-46f6-4d67-96af-d77d3f693623 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:22.724291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6b19e0be-46f6-4d67-96af-d77d3f693623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.733682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 927/1826] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:22.740755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19e0be-46f6-4d67-96af-d77d3f693623 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.743253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6b19e0be-46f6-4d67-96af-d77d3f693623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.757392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19e0be-46f6-4d67-96af-d77d3f693623 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.763562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636fee3-d20e-48e9-84f6-54f241d0f172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.764185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fee3-d20e-48e9-84f6-54f241d0f172 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.765265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fee3-d20e-48e9-84f6-54f241d0f172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.765637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636fee3-d20e-48e9-84f6-54f241d0f172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.766356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636fee3-d20e-48e9-84f6-54f241d0f172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.768595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636fee3-d20e-48e9-84f6-54f241d0f172 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.788032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 928/1827] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:22.790556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657429ba-6f18-427f-8675-bad9362da8e8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.791186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657429ba-6f18-427f-8675-bad9362da8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.791464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657429ba-6f18-427f-8675-bad9362da8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.819274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 900/1828] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:22.819932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868b01e-f4dd-43d5-8f35-8ad5f61f7fce None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.832604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-657429ba-6f18-427f-8675-bad9362da8e8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:22.834102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868b01e-f4dd-43d5-8f35-8ad5f61f7fce None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.834994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-657429ba-6f18-427f-8675-bad9362da8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.839258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2fb04-b0b6-4e6f-80f6-25dc59ae229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:22.848060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 929/1829] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:22.849491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657429ba-6f18-427f-8675-bad9362da8e8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.852790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-657429ba-6f18-427f-8675-bad9362da8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:22.860885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2fb04-b0b6-4e6f-80f6-25dc59ae229a admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:22.861166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2fb04-b0b6-4e6f-80f6-25dc59ae229a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:22.861450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2fb04-b0b6-4e6f-80f6-25dc59ae229a admin admin] PATH_INFO: `/v3/projects/df0e421a1d13496ba5621c57426c7de7/users/5861b9ad9833483bbeb2da57b61bd982/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:22.886780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2fb04-b0b6-4e6f-80f6-25dc59ae229a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0e421a1d13496ba5621c57426c7de7, user_id=5861b9ad9833483bbeb2da57b61bd982, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:22.895730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2fb04-b0b6-4e6f-80f6-25dc59ae229a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:22.923832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 901/1830] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:22 2022] PUT /identity/v3/projects/df0e421a1d13496ba5621c57426c7de7/users/5861b9ad9833483bbeb2da57b61bd982/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:22.925863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2fb04-b0b6-4e6f-80f6-25dc59ae229a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:22.980688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657429ba-6f18-427f-8675-bad9362da8e8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:22.992321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:23.058550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2fb04-b0b6-4e6f-80f6-25dc59ae229a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.065869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810a56d-3c29-4910-b94d-47fdab1acfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.066677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d3456-f55e-4505-a5e8-3f036f9a205b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.067352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d3456-f55e-4505-a5e8-3f036f9a205b service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.067631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d3456-f55e-4505-a5e8-3f036f9a205b service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.067906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d3456-f55e-4505-a5e8-3f036f9a205b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.072476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810a56d-3c29-4910-b94d-47fdab1acfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.072476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810a56d-3c29-4910-b94d-47fdab1acfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.072476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810a56d-3c29-4910-b94d-47fdab1acfeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.072476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810a56d-3c29-4910-b94d-47fdab1acfeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:23.072476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810a56d-3c29-4910-b94d-47fdab1acfeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.073149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810a56d-3c29-4910-b94d-47fdab1acfeb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.104492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 902/1831] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:23.106923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9b3db-74ea-4c23-90f0-f98132ca4692 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.107217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9b3db-74ea-4c23-90f0-f98132ca4692 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.107500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9b3db-74ea-4c23-90f0-f98132ca4692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.152763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bcf9b3db-74ea-4c23-90f0-f98132ca4692 None None] MFA Rules not processed for user `5861b9ad9833483bbeb2da57b61bd982`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:23.154201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bcf9b3db-74ea-4c23-90f0-f98132ca4692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.177357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d3456-f55e-4505-a5e8-3f036f9a205b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.179979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d3456-f55e-4505-a5e8-3f036f9a205b service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.196822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 930/1832] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:23.197724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d3456-f55e-4505-a5e8-3f036f9a205b service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.221026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d3456-f55e-4505-a5e8-3f036f9a205b service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.227739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dfbb1-e80e-46bf-a7d1-0c9f1de77b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.228377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dfbb1-e80e-46bf-a7d1-0c9f1de77b4c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.228739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dfbb1-e80e-46bf-a7d1-0c9f1de77b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.229020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dfbb1-e80e-46bf-a7d1-0c9f1de77b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.229803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dfbb1-e80e-46bf-a7d1-0c9f1de77b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.231967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dfbb1-e80e-46bf-a7d1-0c9f1de77b4c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.269051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 903/1833] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:23.270429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9b3db-74ea-4c23-90f0-f98132ca4692 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.274618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bcf9b3db-74ea-4c23-90f0-f98132ca4692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.280595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 931/1834] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:23.280595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dfbb1-e80e-46bf-a7d1-0c9f1de77b4c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.291918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:23.297082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9b3db-74ea-4c23-90f0-f98132ca4692 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.370493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dfbb1-e80e-46bf-a7d1-0c9f1de77b4c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.377624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2bd16-cf28-4b8f-80d7-37477a8c34b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.378312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2bd16-cf28-4b8f-80d7-37477a8c34b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.378657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2bd16-cf28-4b8f-80d7-37477a8c34b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.378996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2bd16-cf28-4b8f-80d7-37477a8c34b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.379840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2bd16-cf28-4b8f-80d7-37477a8c34b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.382048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2bd16-cf28-4b8f-80d7-37477a8c34b4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.404533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.405223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.405507 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.405812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] PATH_INFO: `/v3/projects/9cf283d0173e4c6f936c5b3a8ce18113` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.407424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf283d0173e4c6f936c5b3a8ce18113)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.409395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.416651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 932/1835] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:23.417096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2bd16-cf28-4b8f-80d7-37477a8c34b4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.473142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf283d0173e4c6f936c5b3a8ce18113', 'request_id': 'req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.480331 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf283d0173e4c6f936c5b3a8ce18113', 'request_id': 'req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.481988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:23.490545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf283d0173e4c6f936c5b3a8ce18113', 'request_id': 'req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.502739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] The token cache is being invalidate because project 9cf283d0173e4c6f936c5b3a8ce18113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:23.503024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.503687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.505239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 904/1836] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:23 2022] DELETE /identity/v3/projects/9cf283d0173e4c6f936c5b3a8ce18113 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:23.505973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.506684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2bd16-cf28-4b8f-80d7-37477a8c34b4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.509921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.583974 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243b706-5f64-455e-b114-5f273da98ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.584529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b706-5f64-455e-b114-5f273da98ddb admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.584814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b706-5f64-455e-b114-5f273da98ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.585090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b706-5f64-455e-b114-5f273da98ddb admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.586318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243b706-5f64-455e-b114-5f273da98ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.588605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243b706-5f64-455e-b114-5f273da98ddb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.615613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 933/1837] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:23.617019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdea43-4e4c-472e-a71d-073bc3e3ddf0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.617199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6b128-f6bd-4242-bffa-078ae06d1c4e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.617331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6b128-f6bd-4242-bffa-078ae06d1c4e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.617772 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6b128-f6bd-4242-bffa-078ae06d1c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.625167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.625903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.626249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.626590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.627290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.629377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.651610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ffa6b128-f6bd-4242-bffa-078ae06d1c4e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:23.653626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ffa6b128-f6bd-4242-bffa-078ae06d1c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.661604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 905/1838] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:23.662778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.665695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.669442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 934/1839] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:23.670509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6b128-f6bd-4242-bffa-078ae06d1c4e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.683054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6b128-f6bd-4242-bffa-078ae06d1c4e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.690340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a1a5a-1548-41a6-925b-f643919f8857 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.691620 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa206c5a-8aed-4225-9ae4-3daaa8c72124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.692347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa206c5a-8aed-4225-9ae4-3daaa8c72124 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.692653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa206c5a-8aed-4225-9ae4-3daaa8c72124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.692926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa206c5a-8aed-4225-9ae4-3daaa8c72124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.693626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa206c5a-8aed-4225-9ae4-3daaa8c72124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.695955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa206c5a-8aed-4225-9ae4-3daaa8c72124 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.696764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.697582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.698037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.698490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.699227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.700979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.710032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 935/1840] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:23.710032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a4d4-bb65-463b-8333-10b0e4e4c607 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.710032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a4d4-bb65-463b-8333-10b0e4e4c607 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.710032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a4d4-bb65-463b-8333-10b0e4e4c607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.714433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 906/1841] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:23.715311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.717103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.731506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313d9e3-61f4-4db8-a845-8e25cd89ea59 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.732585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b4b7a4d4-bb65-463b-8333-10b0e4e4c607 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:23.733771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b4b7a4d4-bb65-463b-8333-10b0e4e4c607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.738140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.739019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.739575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.740281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] PATH_INFO: `/v3/users/4a0dd1194187418aa0fd5ee55e7f4075` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.741509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0dd1194187418aa0fd5ee55e7f4075)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.742304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 936/1842] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:23.743917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.744378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7a4d4-bb65-463b-8333-10b0e4e4c607 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.755381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7a4d4-bb65-463b-8333-10b0e4e4c607 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.769098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb782ae-fbea-4001-9193-30d430e7a278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.769947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb782ae-fbea-4001-9193-30d430e7a278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.770297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb782ae-fbea-4001-9193-30d430e7a278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.770580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb782ae-fbea-4001-9193-30d430e7a278 admin admin] PATH_INFO: `/v3/projects/1233921319a84d188e5f32d2714c7a00/users/f145ade4546f43b5aa516ba702151c07/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.773849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:23.789979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb782ae-fbea-4001-9193-30d430e7a278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1233921319a84d188e5f32d2714c7a00, user_id=f145ade4546f43b5aa516ba702151c07, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.792679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb782ae-fbea-4001-9193-30d430e7a278 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.796520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0dd1194187418aa0fd5ee55e7f4075'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.805428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0dd1194187418aa0fd5ee55e7f4075'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.806480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 937/1843] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:23 2022] PUT /identity/v3/projects/1233921319a84d188e5f32d2714c7a00/users/f145ade4546f43b5aa516ba702151c07/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:23.807280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb782ae-fbea-4001-9193-30d430e7a278 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.815280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0dd1194187418aa0fd5ee55e7f4075'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.815978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0dd1194187418aa0fd5ee55e7f4075'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.816648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0dd1194187418aa0fd5ee55e7f4075'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:23.834647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 907/1844] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:23 2022] DELETE /identity/v3/users/4a0dd1194187418aa0fd5ee55e7f4075 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:23.836131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.837551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.872744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb782ae-fbea-4001-9193-30d430e7a278 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.880455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:INSERT=5,keystone:DELETE=11,keystone:SELECT=84 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:23.906350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ad497-18b6-46e2-9f1f-88ab4e6742e2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.911054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.911663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.911943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.912524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.912919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:23.913221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.916089 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.932690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 908/1845] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:23.932690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.933456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.935463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50b406-d13f-403c-96b0-a73302029bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.936602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50b406-d13f-403c-96b0-a73302029bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.937082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50b406-d13f-403c-96b0-a73302029bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.937523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50b406-d13f-403c-96b0-a73302029bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.938792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50b406-d13f-403c-96b0-a73302029bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.941714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50b406-d13f-403c-96b0-a73302029bf4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.946420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8555e-6641-49bb-90fb-40dcdc244cfc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.951261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5530b-ab6d-44e5-b221-bcd62590ca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.951886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5530b-ab6d-44e5-b221-bcd62590ca8e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.952164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5530b-ab6d-44e5-b221-bcd62590ca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.952456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5530b-ab6d-44e5-b221-bcd62590ca8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.955945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 938/1846] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:23.956083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5530b-ab6d-44e5-b221-bcd62590ca8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.956160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d54f37-f41a-4d9d-980c-0bb6cc3c6124 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.956160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d54f37-f41a-4d9d-980c-0bb6cc3c6124 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.956160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d54f37-f41a-4d9d-980c-0bb6cc3c6124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.956361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5530b-ab6d-44e5-b221-bcd62590ca8e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.966980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 909/1847] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:23.967845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5530b-ab6d-44e5-b221-bcd62590ca8e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.977297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-77d54f37-f41a-4d9d-980c-0bb6cc3c6124 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:23.978034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5530b-ab6d-44e5-b221-bcd62590ca8e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:23.978758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-77d54f37-f41a-4d9d-980c-0bb6cc3c6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:23.982654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4f8d9-98ac-469d-8679-1fec3af6e894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:23.983237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4f8d9-98ac-469d-8679-1fec3af6e894 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:23.983515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4f8d9-98ac-469d-8679-1fec3af6e894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:23.983785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4f8d9-98ac-469d-8679-1fec3af6e894 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:23.984393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4f8d9-98ac-469d-8679-1fec3af6e894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:23.986281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4f8d9-98ac-469d-8679-1fec3af6e894 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:23.986378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 939/1848] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:23.987011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d54f37-f41a-4d9d-980c-0bb6cc3c6124 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:23.988995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-77d54f37-f41a-4d9d-980c-0bb6cc3c6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:24.001212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d54f37-f41a-4d9d-980c-0bb6cc3c6124 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.006322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.007034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.007534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.008544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] PATH_INFO: `/v3/projects/095eaf4a0919496daf09665b402a7c09/users/5794e583b0234d098abec304c567533e/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.015458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 910/1849] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:23 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:24.016964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4f8d9-98ac-469d-8679-1fec3af6e894 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.026171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095eaf4a0919496daf09665b402a7c09, user_id=5794e583b0234d098abec304c567533e, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.028932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.046372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 940/1850] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:23 2022] PUT /identity/v3/projects/095eaf4a0919496daf09665b402a7c09/users/5794e583b0234d098abec304c567533e/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:24.047436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.049202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:24.085037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4f8d9-98ac-469d-8679-1fec3af6e894 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.115220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4ec7-092f-4425-8552-5c5a7e63ef02 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.120223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749604e2-7ad0-4e8e-84a9-4bce8d13bac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.120748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749604e2-7ad0-4e8e-84a9-4bce8d13bac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.121033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749604e2-7ad0-4e8e-84a9-4bce8d13bac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.121307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749604e2-7ad0-4e8e-84a9-4bce8d13bac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.130184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:24.158725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da39a7d-df79-4fd5-b616-50534787c9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.159491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39a7d-df79-4fd5-b616-50534787c9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.159958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39a7d-df79-4fd5-b616-50534787c9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.160476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39a7d-df79-4fd5-b616-50534787c9ce admin admin] PATH_INFO: `/v3/projects/1233921319a84d188e5f32d2714c7a00/users/f145ade4546f43b5aa516ba702151c07/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.162392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da39a7d-df79-4fd5-b616-50534787c9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1233921319a84d188e5f32d2714c7a00, user_id=f145ade4546f43b5aa516ba702151c07, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.165391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da39a7d-df79-4fd5-b616-50534787c9ce admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.182962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 911/1851] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:24 2022] PUT /identity/v3/projects/1233921319a84d188e5f32d2714c7a00/users/f145ade4546f43b5aa516ba702151c07/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:24.183816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da39a7d-df79-4fd5-b616-50534787c9ce admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.198610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749604e2-7ad0-4e8e-84a9-4bce8d13bac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.200719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749604e2-7ad0-4e8e-84a9-4bce8d13bac1 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.214919 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 941/1852] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:24.216542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc563bc-b05f-4b9e-a2eb-d41ca999e50a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.216955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc563bc-b05f-4b9e-a2eb-d41ca999e50a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.217442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc563bc-b05f-4b9e-a2eb-d41ca999e50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.236337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:24.282377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da39a7d-df79-4fd5-b616-50534787c9ce admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.284414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-afc563bc-b05f-4b9e-a2eb-d41ca999e50a None None] MFA Rules not processed for user `5794e583b0234d098abec304c567533e`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:24.291183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-afc563bc-b05f-4b9e-a2eb-d41ca999e50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:24.294207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b035f4-293b-4cf7-b854-c7bf946a1083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.294952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b035f4-293b-4cf7-b854-c7bf946a1083 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.295116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b035f4-293b-4cf7-b854-c7bf946a1083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.295534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b035f4-293b-4cf7-b854-c7bf946a1083 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.296171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b035f4-293b-4cf7-b854-c7bf946a1083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.300977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b035f4-293b-4cf7-b854-c7bf946a1083 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.359917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 912/1853] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:24 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:24.360961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b6087-46ae-4bae-852a-99369a9e8e88 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.361242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b6087-46ae-4bae-852a-99369a9e8e88 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.361716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b6087-46ae-4bae-852a-99369a9e8e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.421146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5c4b6087-46ae-4bae-852a-99369a9e8e88 None None] MFA Rules not processed for user `f145ade4546f43b5aa516ba702151c07`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:24.430820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5c4b6087-46ae-4bae-852a-99369a9e8e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:24.455620 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 942/1854] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:24.459699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc563bc-b05f-4b9e-a2eb-d41ca999e50a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.477603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc563bc-b05f-4b9e-a2eb-d41ca999e50a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.489170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca7e4d-e8a3-4c0e-a1e4-4a71985b961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.489892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca7e4d-e8a3-4c0e-a1e4-4a71985b961a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.490182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca7e4d-e8a3-4c0e-a1e4-4a71985b961a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.490464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca7e4d-e8a3-4c0e-a1e4-4a71985b961a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.491125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca7e4d-e8a3-4c0e-a1e4-4a71985b961a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.493486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca7e4d-e8a3-4c0e-a1e4-4a71985b961a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.528796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 943/1855] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:24.533031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca7e4d-e8a3-4c0e-a1e4-4a71985b961a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.567118 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca7e4d-e8a3-4c0e-a1e4-4a71985b961a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.573477 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.574382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.574810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.575229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.576074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.579678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.592653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 913/1856] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:24.593513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b6087-46ae-4bae-852a-99369a9e8e88 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.625726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b6087-46ae-4bae-852a-99369a9e8e88 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.633482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 944/1857] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:24 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:24.634484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.636256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:24.644389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38dad7-a317-48d6-8c51-7f9cad5a31a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.645338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38dad7-a317-48d6-8c51-7f9cad5a31a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.645718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38dad7-a317-48d6-8c51-7f9cad5a31a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.646166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38dad7-a317-48d6-8c51-7f9cad5a31a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.647071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38dad7-a317-48d6-8c51-7f9cad5a31a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.649641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38dad7-a317-48d6-8c51-7f9cad5a31a3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.666665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 914/1858] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:24.666665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38dad7-a317-48d6-8c51-7f9cad5a31a3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.680208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38dad7-a317-48d6-8c51-7f9cad5a31a3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.775369 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d668eaf-46a6-4d91-bf77-efedef15959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.776161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d668eaf-46a6-4d91-bf77-efedef15959c admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.776614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d668eaf-46a6-4d91-bf77-efedef15959c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.776990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d668eaf-46a6-4d91-bf77-efedef15959c admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.777730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d668eaf-46a6-4d91-bf77-efedef15959c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.779786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d668eaf-46a6-4d91-bf77-efedef15959c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.803412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad76b2-06d2-407e-9bfa-e30fff8445f1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.808661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.809525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.809951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.810578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.825201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 915/1859] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:24 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:24.825201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d668eaf-46a6-4d91-bf77-efedef15959c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.859585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d668eaf-46a6-4d91-bf77-efedef15959c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:24.861575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:24.872969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc32e8b-86dd-4d92-a795-847afa4df3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:24.874320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc32e8b-86dd-4d92-a795-847afa4df3de admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:24.875246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc32e8b-86dd-4d92-a795-847afa4df3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:24.875816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc32e8b-86dd-4d92-a795-847afa4df3de admin admin] PATH_INFO: `/v3/projects/fbbba6ffd7864c80b3d3aea108547156/users/1e819f8947f64b91918fa747354f3075/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:24.896414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc32e8b-86dd-4d92-a795-847afa4df3de admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbba6ffd7864c80b3d3aea108547156, user_id=1e819f8947f64b91918fa747354f3075, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:24.899494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc32e8b-86dd-4d92-a795-847afa4df3de admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:24.917018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 916/1860] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:24 2022] PUT /identity/v3/projects/fbbba6ffd7864c80b3d3aea108547156/users/1e819f8947f64b91918fa747354f3075/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:24.918039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc32e8b-86dd-4d92-a795-847afa4df3de admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:24.940487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:25.010098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.012419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.036678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 945/1861] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:25.037869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.039956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.046214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc32e8b-86dd-4d92-a795-847afa4df3de admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.055397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970e5cb-badf-4ffc-b68e-41a42b531453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.056250 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970e5cb-badf-4ffc-b68e-41a42b531453 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.056922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970e5cb-badf-4ffc-b68e-41a42b531453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.057316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970e5cb-badf-4ffc-b68e-41a42b531453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.060474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970e5cb-badf-4ffc-b68e-41a42b531453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.060474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970e5cb-badf-4ffc-b68e-41a42b531453 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.067711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36dad-710e-4c15-883a-e2c472db2198 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.085383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 917/1862] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:25.085383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7955-6e7a-4554-9032-acb0d64f5eb4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.085383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7955-6e7a-4554-9032-acb0d64f5eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.085383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7955-6e7a-4554-9032-acb0d64f5eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.085653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.085653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.085653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.085653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] PATH_INFO: `/v3/projects/c224332f55774723b08996c61bf3186a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.085929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c224332f55774723b08996c61bf3186a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.085929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.119706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-067c7955-6e7a-4554-9032-acb0d64f5eb4 None None] MFA Rules not processed for user `1e819f8947f64b91918fa747354f3075`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:25.123687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-067c7955-6e7a-4554-9032-acb0d64f5eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.170515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c224332f55774723b08996c61bf3186a', 'request_id': 'req-63f76cd0-a8bb-4725-a65d-b4010ada7db9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.176825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c224332f55774723b08996c61bf3186a', 'request_id': 'req-63f76cd0-a8bb-4725-a65d-b4010ada7db9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.185798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c224332f55774723b08996c61bf3186a', 'request_id': 'req-63f76cd0-a8bb-4725-a65d-b4010ada7db9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.198276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] The token cache is being invalidate because project c224332f55774723b08996c61bf3186a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:25.198556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.199214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.200751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 946/1863] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:25 2022] DELETE /identity/v3/projects/c224332f55774723b08996c61bf3186a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:25.201502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.205018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.219192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 918/1864] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:25.220131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c7955-6e7a-4554-9032-acb0d64f5eb4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.221553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-067c7955-6e7a-4554-9032-acb0d64f5eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.273290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f76cd0-a8bb-4725-a65d-b4010ada7db9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.278135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.278793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.279072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.279349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] PATH_INFO: `/v3/projects/4c1bec9f5c03472d94fbe7ba748a3a98/users/abfa6d436339409c98da3bdf4b5767c7/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.281141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c7955-6e7a-4554-9032-acb0d64f5eb4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.294072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1bec9f5c03472d94fbe7ba748a3a98, user_id=abfa6d436339409c98da3bdf4b5767c7, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.296774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.310494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 947/1865] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:25 2022] PUT /identity/v3/projects/4c1bec9f5c03472d94fbe7ba748a3a98/users/abfa6d436339409c98da3bdf4b5767c7/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:25.311318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.312889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.333551 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:25.336382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.337017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.337297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.337571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.339397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.344747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:25.385940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c42d9-c924-4cbc-9cb1-7250cad81996 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.391391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.392028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.392310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.392731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] PATH_INFO: `/v3/users/1819c9ce24f94b14af8b03ffcfcf343b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.394086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1819c9ce24f94b14af8b03ffcfcf343b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.395843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.414569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.416010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.435963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 919/1866] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:25.436836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.438934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.452153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1819c9ce24f94b14af8b03ffcfcf343b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.459238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb3348-ca49-44da-a6f0-cb88702f3fc0 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.460608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1819c9ce24f94b14af8b03ffcfcf343b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.467126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.467390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.467675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.467951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.468725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.471425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.479798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1819c9ce24f94b14af8b03ffcfcf343b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.483033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1819c9ce24f94b14af8b03ffcfcf343b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.483835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1819c9ce24f94b14af8b03ffcfcf343b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:25.508739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 920/1867] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:25 2022] POST /identity/v3/projects => generated 391 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:25.510064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.511746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.525324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 948/1868] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:25 2022] DELETE /identity/v3/users/1819c9ce24f94b14af8b03ffcfcf343b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:25.526324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.527992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.590381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5778b26-547e-41fc-9b23-697810626d0f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.648309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c8847-22d6-4f94-8824-13ab4febf343 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.655522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bde279-270c-4e95-b283-f661e6bea97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.656476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bde279-270c-4e95-b283-f661e6bea97d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.656950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bde279-270c-4e95-b283-f661e6bea97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.657246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bde279-270c-4e95-b283-f661e6bea97d admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.657967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bde279-270c-4e95-b283-f661e6bea97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.660175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bde279-270c-4e95-b283-f661e6bea97d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.662626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4206c04-b05d-4b14-948b-808d8a2f60d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.663242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206c04-b05d-4b14-948b-808d8a2f60d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.663537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206c04-b05d-4b14-948b-808d8a2f60d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.663964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206c04-b05d-4b14-948b-808d8a2f60d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.665413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4206c04-b05d-4b14-948b-808d8a2f60d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.667112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4206c04-b05d-4b14-948b-808d8a2f60d0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.681057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 921/1869] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:25.685239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4206c04-b05d-4b14-948b-808d8a2f60d0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.689011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 949/1870] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:25 2022] POST /identity/v3/users => generated 302 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:25.692190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bde279-270c-4e95-b283-f661e6bea97d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.701188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4206c04-b05d-4b14-948b-808d8a2f60d0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.704819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bde279-270c-4e95-b283-f661e6bea97d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.706365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1feeb90-6ace-4c4a-ac65-97c6aaa58364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.707173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feeb90-6ace-4c4a-ac65-97c6aaa58364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.707458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feeb90-6ace-4c4a-ac65-97c6aaa58364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.707774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feeb90-6ace-4c4a-ac65-97c6aaa58364 admin admin] PATH_INFO: `/v3/projects/4c1bec9f5c03472d94fbe7ba748a3a98/users/abfa6d436339409c98da3bdf4b5767c7/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.709862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1feeb90-6ace-4c4a-ac65-97c6aaa58364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1bec9f5c03472d94fbe7ba748a3a98, user_id=abfa6d436339409c98da3bdf4b5767c7, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.711849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989b3bc-fdb0-4d10-830d-dde706fc071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.712670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989b3bc-fdb0-4d10-830d-dde706fc071c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.713095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989b3bc-fdb0-4d10-830d-dde706fc071c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.713509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989b3bc-fdb0-4d10-830d-dde706fc071c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.714180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1feeb90-6ace-4c4a-ac65-97c6aaa58364 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.714951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989b3bc-fdb0-4d10-830d-dde706fc071c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.717140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989b3bc-fdb0-4d10-830d-dde706fc071c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.740530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 922/1871] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:25 2022] PUT /identity/v3/projects/4c1bec9f5c03472d94fbe7ba748a3a98/users/abfa6d436339409c98da3bdf4b5767c7/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:25.748222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 950/1872] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:25.749494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670958c-f40c-4bd2-9a26-37d1913d4d00 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.749881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670958c-f40c-4bd2-9a26-37d1913d4d00 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.750170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670958c-f40c-4bd2-9a26-37d1913d4d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.752759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989b3bc-fdb0-4d10-830d-dde706fc071c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.764489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989b3bc-fdb0-4d10-830d-dde706fc071c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:25.790451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:25.802198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:25.809392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2670958c-f40c-4bd2-9a26-37d1913d4d00 None None] MFA Rules not processed for user `abfa6d436339409c98da3bdf4b5767c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:25.813105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2670958c-f40c-4bd2-9a26-37d1913d4d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.881666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:25.883121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:25.883571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:25.884001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] PATH_INFO: `/v3/projects/1aa47b5ec5ce48ecaeb72e1c61fe50c7/users/07f4bea9d19d4777b836db94659edc18/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:25.912598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa47b5ec5ce48ecaeb72e1c61fe50c7, user_id=07f4bea9d19d4777b836db94659edc18, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:25.918948 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:25.946515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 951/1873] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:25 2022] PUT /identity/v3/projects/1aa47b5ec5ce48ecaeb72e1c61fe50c7/users/07f4bea9d19d4777b836db94659edc18/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:25.947868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.959794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:25.974120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 923/1874] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:25.975499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670958c-f40c-4bd2-9a26-37d1913d4d00 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:25.995698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670958c-f40c-4bd2-9a26-37d1913d4d00 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.060646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c45af-0375-4e6a-9ebb-fd5ad5e8dc3b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.062746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.063400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.063703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.063977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.064650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.065796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.066255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.066556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.066903 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.067244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] PATH_INFO: `/v3/projects/1064052adad347dd92c493818334bdfd` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.068565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1064052adad347dd92c493818334bdfd)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.070713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.076454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 924/1875] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:26.077108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.080392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.127196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1064052adad347dd92c493818334bdfd', 'request_id': 'req-14d56eb1-67eb-496e-92ad-316be390f9b5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:26.134109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1064052adad347dd92c493818334bdfd', 'request_id': 'req-14d56eb1-67eb-496e-92ad-316be390f9b5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:26.142794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1064052adad347dd92c493818334bdfd', 'request_id': 'req-14d56eb1-67eb-496e-92ad-316be390f9b5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:26.144040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003a487-3e09-44c8-9421-ea508cf14f28 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.157027 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] The token cache is being invalidate because project 1064052adad347dd92c493818334bdfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:26.157315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:26.158026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:26.163972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 952/1876] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:25 2022] DELETE /identity/v3/projects/1064052adad347dd92c493818334bdfd => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:26.163972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.163972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.199223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:26.209800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cceee-7780-43bf-88fd-3c97335ac05f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.211799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cceee-7780-43bf-88fd-3c97335ac05f service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.214070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cceee-7780-43bf-88fd-3c97335ac05f service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.214453 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cceee-7780-43bf-88fd-3c97335ac05f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.216713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1f8cceee-7780-43bf-88fd-3c97335ac05f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.222286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:26.240604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d56eb1-67eb-496e-92ad-316be390f9b5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.246196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021d895-2c4f-48a6-9f0e-062f29bf432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.246864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d895-2c4f-48a6-9f0e-062f29bf432f admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.247222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d895-2c4f-48a6-9f0e-062f29bf432f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.247417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021d895-2c4f-48a6-9f0e-062f29bf432f admin admin] PATH_INFO: `/v3/projects/1aa47b5ec5ce48ecaeb72e1c61fe50c7/users/07f4bea9d19d4777b836db94659edc18/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.249560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021d895-2c4f-48a6-9f0e-062f29bf432f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa47b5ec5ce48ecaeb72e1c61fe50c7, user_id=07f4bea9d19d4777b836db94659edc18, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.257487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021d895-2c4f-48a6-9f0e-062f29bf432f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.271235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 953/1877] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:26 2022] PUT /identity/v3/projects/1aa47b5ec5ce48ecaeb72e1c61fe50c7/users/07f4bea9d19d4777b836db94659edc18/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:26.273314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015950d-9db2-4efe-b465-50c26a150ba1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.273796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015950d-9db2-4efe-b465-50c26a150ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.274262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015950d-9db2-4efe-b465-50c26a150ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.290855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cceee-7780-43bf-88fd-3c97335ac05f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.292207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8cceee-7780-43bf-88fd-3c97335ac05f service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.299695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5015950d-9db2-4efe-b465-50c26a150ba1 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:26.301217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5015950d-9db2-4efe-b465-50c26a150ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.301578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 925/1878] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:26.302852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8d82-4c2a-4732-b39b-091108016b62 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.303134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8d82-4c2a-4732-b39b-091108016b62 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.303403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8d82-4c2a-4732-b39b-091108016b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.332671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0a8f8d82-4c2a-4732-b39b-091108016b62 None None] MFA Rules not processed for user `07f4bea9d19d4777b836db94659edc18`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:26.333934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0a8f8d82-4c2a-4732-b39b-091108016b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.368709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 954/1879] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:26.376322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4c66-ddf1-4b9c-9a61-6b978a4b528d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.376635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4c66-ddf1-4b9c-9a61-6b978a4b528d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.377513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a4c66-ddf1-4b9c-9a61-6b978a4b528d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.402865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-705a4c66-ddf1-4b9c-9a61-6b978a4b528d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:26.404328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-705a4c66-ddf1-4b9c-9a61-6b978a4b528d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.412920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 955/1880] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:26.418318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a4c66-ddf1-4b9c-9a61-6b978a4b528d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.421031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 926/1881] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:26.432833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a4c66-ddf1-4b9c-9a61-6b978a4b528d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.436488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f8d82-4c2a-4732-b39b-091108016b62 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.437793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.438398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.438677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.438801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0a8f8d82-4c2a-4732-b39b-091108016b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.438947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.439648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:26.440022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.441818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.455351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 956/1882] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:14:26.459798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.471409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a736aa6-89e8-4ccb-b6a3-cd0692c199d5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.476978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd866da-bc56-40a2-a7f9-2cce1a49e112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.477660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd866da-bc56-40a2-a7f9-2cce1a49e112 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.477965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd866da-bc56-40a2-a7f9-2cce1a49e112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.478238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd866da-bc56-40a2-a7f9-2cce1a49e112 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.478892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd866da-bc56-40a2-a7f9-2cce1a49e112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.480840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd866da-bc56-40a2-a7f9-2cce1a49e112 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.507005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f8d82-4c2a-4732-b39b-091108016b62 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.510841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.511469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.511750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.512024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.515839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 957/1883] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:26 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:26.521343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd866da-bc56-40a2-a7f9-2cce1a49e112 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.522426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.524769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.535349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 927/1884] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:26.635781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.637301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.646577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd866da-bc56-40a2-a7f9-2cce1a49e112 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.654670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c884f-cff7-4f4b-85f5-720c9f14da17 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.661030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589f7f7-1e03-41bb-af90-5da967ede729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.661622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589f7f7-1e03-41bb-af90-5da967ede729 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.661905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589f7f7-1e03-41bb-af90-5da967ede729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.664670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589f7f7-1e03-41bb-af90-5da967ede729 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.665777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589f7f7-1e03-41bb-af90-5da967ede729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.667777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589f7f7-1e03-41bb-af90-5da967ede729 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.693893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:26.712928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 958/1885] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:26 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:26.713849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589f7f7-1e03-41bb-af90-5da967ede729 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.730485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589f7f7-1e03-41bb-af90-5da967ede729 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.731837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395835c2-3fb2-4fcd-9cec-b6cb7e60cd2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.733120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395835c2-3fb2-4fcd-9cec-b6cb7e60cd2b service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.733951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395835c2-3fb2-4fcd-9cec-b6cb7e60cd2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.734531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395835c2-3fb2-4fcd-9cec-b6cb7e60cd2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.735471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df671569-655b-40de-bd18-97197f87a5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.736074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df671569-655b-40de-bd18-97197f87a5df admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.736358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df671569-655b-40de-bd18-97197f87a5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.736665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df671569-655b-40de-bd18-97197f87a5df admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.737316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df671569-655b-40de-bd18-97197f87a5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.738161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-395835c2-3fb2-4fcd-9cec-b6cb7e60cd2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.738917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df671569-655b-40de-bd18-97197f87a5df admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.745362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:26.750851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 959/1886] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:26.755141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df671569-655b-40de-bd18-97197f87a5df admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.767555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df671569-655b-40de-bd18-97197f87a5df admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.774946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55440347-539f-4a97-a1e5-6ff1283d8bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.775632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55440347-539f-4a97-a1e5-6ff1283d8bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.775915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55440347-539f-4a97-a1e5-6ff1283d8bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.776191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55440347-539f-4a97-a1e5-6ff1283d8bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.776869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55440347-539f-4a97-a1e5-6ff1283d8bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.778632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55440347-539f-4a97-a1e5-6ff1283d8bf3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.792546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 960/1887] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:26.796711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55440347-539f-4a97-a1e5-6ff1283d8bf3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:26.807665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55440347-539f-4a97-a1e5-6ff1283d8bf3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:26.809181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395835c2-3fb2-4fcd-9cec-b6cb7e60cd2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.810620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395835c2-3fb2-4fcd-9cec-b6cb7e60cd2b service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.812277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6a2f0-d407-4c6c-8154-f123a81e412c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:26.812947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6a2f0-d407-4c6c-8154-f123a81e412c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.813226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6a2f0-d407-4c6c-8154-f123a81e412c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.813535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6a2f0-d407-4c6c-8154-f123a81e412c admin admin] PATH_INFO: `/v3/projects/9476e500c8094d86bc8b3b9006b66b5c/users/b623534e31054ee1987eca9a2345f65e/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.819415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 928/1888] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:26.829225 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6a2f0-d407-4c6c-8154-f123a81e412c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9476e500c8094d86bc8b3b9006b66b5c, user_id=b623534e31054ee1987eca9a2345f65e, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:26.832196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6a2f0-d407-4c6c-8154-f123a81e412c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:26.846744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 961/1889] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:26 2022] PUT /identity/v3/projects/9476e500c8094d86bc8b3b9006b66b5c/users/b623534e31054ee1987eca9a2345f65e/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:26.857645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966136-cbe4-45b3-9c69-c517f4cf5176 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:26.857927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966136-cbe4-45b3-9c69-c517f4cf5176 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:26.858200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966136-cbe4-45b3-9c69-c517f4cf5176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:26.887713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-07966136-cbe4-45b3-9c69-c517f4cf5176 None None] MFA Rules not processed for user `b623534e31054ee1987eca9a2345f65e`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:26.888939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-07966136-cbe4-45b3-9c69-c517f4cf5176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:26.971992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 929/1890] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:26.983133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6a2f0-d407-4c6c-8154-f123a81e412c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:27.045627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6a2f0-d407-4c6c-8154-f123a81e412c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:27.049797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219c30c-565f-43fe-9300-430843c946da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:27.050700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219c30c-565f-43fe-9300-430843c946da service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:27.050980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219c30c-565f-43fe-9300-430843c946da service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:27.051264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219c30c-565f-43fe-9300-430843c946da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:27.064088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219c30c-565f-43fe-9300-430843c946da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:27.065645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219c30c-565f-43fe-9300-430843c946da service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:27.091938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 962/1891] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:27.107856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07966136-cbe4-45b3-9c69-c517f4cf5176 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:27.200226 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07966136-cbe4-45b3-9c69-c517f4cf5176 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:27.204952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:27.205586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:27.205862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:27.206134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] PATH_INFO: `/v3/users/b623534e31054ee1987eca9a2345f65e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:27.207290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b623534e31054ee1987eca9a2345f65e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:27.209268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:27.232042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:27.260211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b623534e31054ee1987eca9a2345f65e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:27.268061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b623534e31054ee1987eca9a2345f65e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:27.275584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b623534e31054ee1987eca9a2345f65e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:27.276214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b623534e31054ee1987eca9a2345f65e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:27.276838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b623534e31054ee1987eca9a2345f65e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:27.296723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 930/1892] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:27 2022] DELETE /identity/v3/users/b623534e31054ee1987eca9a2345f65e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:27.301710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6523e8-999f-4d46-b539-387c523a7aeb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:27.302003 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6523e8-999f-4d46-b539-387c523a7aeb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:27.302282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6523e8-999f-4d46-b539-387c523a7aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:27.321821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-da6523e8-999f-4d46-b539-387c523a7aeb None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:27.323015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-da6523e8-999f-4d46-b539-387c523a7aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:27.381553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 963/1893] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:27.392739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:27.394709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:27.461043 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bf194-edd5-4fbc-8e36-e205dc19b66e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:27.464655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c5e0b-8a57-4e2b-bfee-df033bd49516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:27.465263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c5e0b-8a57-4e2b-bfee-df033bd49516 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:27.465651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c5e0b-8a57-4e2b-bfee-df033bd49516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:27.465922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c5e0b-8a57-4e2b-bfee-df033bd49516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:27.474969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c5e0b-8a57-4e2b-bfee-df033bd49516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:27.476796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c5e0b-8a57-4e2b-bfee-df033bd49516 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:27.487163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 931/1894] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:28.186270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6523e8-999f-4d46-b539-387c523a7aeb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:28.187903 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-da6523e8-999f-4d46-b539-387c523a7aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:28.216920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6523e8-999f-4d46-b539-387c523a7aeb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:28.222120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:28.222751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.223028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.223299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] PATH_INFO: `/v3/projects/9476e500c8094d86bc8b3b9006b66b5c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.224657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9476e500c8094d86bc8b3b9006b66b5c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:28.227297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:28.305630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9476e500c8094d86bc8b3b9006b66b5c', 'request_id': 'req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:28.312401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9476e500c8094d86bc8b3b9006b66b5c', 'request_id': 'req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:28.316205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:28.321199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9476e500c8094d86bc8b3b9006b66b5c', 'request_id': 'req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:28.335850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] The token cache is being invalidate because project 9476e500c8094d86bc8b3b9006b66b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:28.336189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:28.337076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64c061a8-71ee-4df5-9f13-ab51fa9e8f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:28.342076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 964/1895] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:28 2022] DELETE /identity/v3/projects/9476e500c8094d86bc8b3b9006b66b5c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:28.343918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18e3f3-f566-44b4-a3a2-c9578aa0b1e7 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.344827 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18e3f3-f566-44b4-a3a2-c9578aa0b1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.345208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18e3f3-f566-44b4-a3a2-c9578aa0b1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.375608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2f18e3f3-f566-44b4-a3a2-c9578aa0b1e7 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:28.377030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2f18e3f3-f566-44b4-a3a2-c9578aa0b1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:28.462738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 932/1896] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:28.480821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d82cd-2876-4133-8c06-7364527f1d15 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.481161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d82cd-2876-4133-8c06-7364527f1d15 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.481430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d82cd-2876-4133-8c06-7364527f1d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.503949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3b7d82cd-2876-4133-8c06-7364527f1d15 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:28.505382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3b7d82cd-2876-4133-8c06-7364527f1d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:28.514014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 965/1897] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:28.518528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18e3f3-f566-44b4-a3a2-c9578aa0b1e7 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:28.529593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18e3f3-f566-44b4-a3a2-c9578aa0b1e7 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:28.534544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:28.535163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.535444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.535718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.536429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:28.536770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:28.539134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:28.551867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 933/1898] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:28.556993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d82cd-2876-4133-8c06-7364527f1d15 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:28.570520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d82cd-2876-4133-8c06-7364527f1d15 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:28.575242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910fb08-6272-4953-a032-d6217fa8e54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:28.575876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910fb08-6272-4953-a032-d6217fa8e54d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.576125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910fb08-6272-4953-a032-d6217fa8e54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.576396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910fb08-6272-4953-a032-d6217fa8e54d admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.577058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910fb08-6272-4953-a032-d6217fa8e54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:28.578951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910fb08-6272-4953-a032-d6217fa8e54d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:28.623760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 966/1899] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:28 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:28.626039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:28.637844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf82082-c943-46aa-bdcc-e2505f830746 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:28.666703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:28.711767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58c82f-9325-4bd0-88ea-4f99a01a6c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:28.712765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58c82f-9325-4bd0-88ea-4f99a01a6c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.713060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58c82f-9325-4bd0-88ea-4f99a01a6c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.713468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58c82f-9325-4bd0-88ea-4f99a01a6c7e admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.714150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58c82f-9325-4bd0-88ea-4f99a01a6c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:28.716185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58c82f-9325-4bd0-88ea-4f99a01a6c7e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:28.743998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 934/1900] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:28 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:28.749087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910fb08-6272-4953-a032-d6217fa8e54d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:28.767111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910fb08-6272-4953-a032-d6217fa8e54d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:28.774899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7881e8-4786-464b-9ebb-67da33a11d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:28.775611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7881e8-4786-464b-9ebb-67da33a11d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.776093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7881e8-4786-464b-9ebb-67da33a11d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.776463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7881e8-4786-464b-9ebb-67da33a11d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.777262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7881e8-4786-464b-9ebb-67da33a11d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:28.778933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7881e8-4786-464b-9ebb-67da33a11d1b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:28.800731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 967/1901] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:28.802089 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58c82f-9325-4bd0-88ea-4f99a01a6c7e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:28.814171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58c82f-9325-4bd0-88ea-4f99a01a6c7e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:28.822107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97027ac8-f541-490d-adaf-d952d23ec25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:28.822736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027ac8-f541-490d-adaf-d952d23ec25f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.823015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027ac8-f541-490d-adaf-d952d23ec25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.823288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027ac8-f541-490d-adaf-d952d23ec25f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.823948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97027ac8-f541-490d-adaf-d952d23ec25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:28.825908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97027ac8-f541-490d-adaf-d952d23ec25f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:28.864494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 935/1902] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:28.869787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7881e8-4786-464b-9ebb-67da33a11d1b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:28.881253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7881e8-4786-464b-9ebb-67da33a11d1b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:28.889004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:28.891197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.891580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.892373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] PATH_INFO: `/v3/projects/9445777028a94a0386b4006db80049ce/users/a002c6d8462f47929885473d594a77a9/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.911011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9445777028a94a0386b4006db80049ce, user_id=a002c6d8462f47929885473d594a77a9, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:28.915047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:28.937869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 968/1903] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:28 2022] PUT /identity/v3/projects/9445777028a94a0386b4006db80049ce/users/a002c6d8462f47929885473d594a77a9/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:28.953987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4b88-fed3-402e-9401-691360e6f8d6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:28.955760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4b88-fed3-402e-9401-691360e6f8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:28.956254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4b88-fed3-402e-9401-691360e6f8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:28.996257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-511f4b88-fed3-402e-9401-691360e6f8d6 None None] MFA Rules not processed for user `a002c6d8462f47929885473d594a77a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:28.997603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-511f4b88-fed3-402e-9401-691360e6f8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:29.098853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 936/1904] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:29.222136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:29.225314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:29.316809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9dd20-45d5-432d-b572-dcdf26825c2c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:29.321527 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:29.322244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:29.322509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:29.322828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:29.333424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:29.334886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:29.345396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 969/1905] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:32.354157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f4b88-fed3-402e-9401-691360e6f8d6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:32.368551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f4b88-fed3-402e-9401-691360e6f8d6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:32.373224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74abd1-45cc-4fc5-8d98-2afdf0b407a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:32.373865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74abd1-45cc-4fc5-8d98-2afdf0b407a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:32.374145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74abd1-45cc-4fc5-8d98-2afdf0b407a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:32.374455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74abd1-45cc-4fc5-8d98-2afdf0b407a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:32.377335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2c74abd1-45cc-4fc5-8d98-2afdf0b407a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:32.478801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74abd1-45cc-4fc5-8d98-2afdf0b407a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:32.480353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74abd1-45cc-4fc5-8d98-2afdf0b407a0 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:32.490712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 937/1906] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:32.691687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:32.700347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:32.794958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813a3c8-fd9d-48db-8798-3b5445b8ea8e service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:32.802237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:32.803046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:32.803474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:32.804761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] PATH_INFO: `/v3/users/1e819f8947f64b91918fa747354f3075` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:32.807321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e819f8947f64b91918fa747354f3075)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:32.810565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:32.868605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:32.912941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e819f8947f64b91918fa747354f3075'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:32.938412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e819f8947f64b91918fa747354f3075'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:32.963309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e819f8947f64b91918fa747354f3075'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:32.964765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e819f8947f64b91918fa747354f3075'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:32.966335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e819f8947f64b91918fa747354f3075'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.032859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 970/1907] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:32 2022] DELETE /identity/v3/users/1e819f8947f64b91918fa747354f3075 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:33.034249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2f97-5912-4eb8-a9ae-0dd4b4940efe None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:33.034536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2f97-5912-4eb8-a9ae-0dd4b4940efe None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:33.034836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2f97-5912-4eb8-a9ae-0dd4b4940efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:33.080293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-345a2f97-5912-4eb8-a9ae-0dd4b4940efe None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:33.081562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-345a2f97-5912-4eb8-a9ae-0dd4b4940efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:33.133622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:33.223440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 938/1908] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:33.247842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:33.250779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:33.255776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a2f97-5912-4eb8-a9ae-0dd4b4940efe None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:33.258356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-345a2f97-5912-4eb8-a9ae-0dd4b4940efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:33.283170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a2f97-5912-4eb8-a9ae-0dd4b4940efe None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:33.290853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:33.291759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:33.293344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:33.293344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] PATH_INFO: `/v3/users/5861b9ad9833483bbeb2da57b61bd982` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:33.294470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5861b9ad9833483bbeb2da57b61bd982)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:33.297234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:33.378410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5861b9ad9833483bbeb2da57b61bd982'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.387633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5861b9ad9833483bbeb2da57b61bd982'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.390863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a4823-c9a8-40ff-81a8-e602eb0fa35c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:33.395964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5861b9ad9833483bbeb2da57b61bd982'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.396770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5861b9ad9833483bbeb2da57b61bd982'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.397421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c770d53-2543-49b2-bf24-83dbc9df24e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5861b9ad9833483bbeb2da57b61bd982'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.400740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:33.401368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:33.401642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:33.401905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:33.403915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:33.419548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:33.421917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:33.424105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 939/1909] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:33 2022] DELETE /identity/v3/users/5861b9ad9833483bbeb2da57b61bd982 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:33.429625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f4e7f-e41d-495a-97cc-65afe929709e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:33.429961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f4e7f-e41d-495a-97cc-65afe929709e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:33.430391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f4e7f-e41d-495a-97cc-65afe929709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:33.442505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 971/1910] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:33.457651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-634f4e7f-e41d-495a-97cc-65afe929709e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:33.458882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-634f4e7f-e41d-495a-97cc-65afe929709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:33.545282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 940/1911] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:33.556187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:33.557686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:33.568931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e2f53-cc4f-4d24-896c-f9964dbda11d service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:33.625340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7222d7-b6b3-497d-9ae7-181af7a21c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:33.626101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7222d7-b6b3-497d-9ae7-181af7a21c6d service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:33.626379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7222d7-b6b3-497d-9ae7-181af7a21c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:33.626655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7222d7-b6b3-497d-9ae7-181af7a21c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:33.637293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7222d7-b6b3-497d-9ae7-181af7a21c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:33.638826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7222d7-b6b3-497d-9ae7-181af7a21c6d service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:33.649630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 972/1912] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:33.739992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f4e7f-e41d-495a-97cc-65afe929709e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:33.742439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-634f4e7f-e41d-495a-97cc-65afe929709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:33.759906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f4e7f-e41d-495a-97cc-65afe929709e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:33.766769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:33.767422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:33.768312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:33.768635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] PATH_INFO: `/v3/projects/fbbba6ffd7864c80b3d3aea108547156` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:33.769790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbbba6ffd7864c80b3d3aea108547156)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:33.771696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:33.821158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:33.832917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbbba6ffd7864c80b3d3aea108547156', 'request_id': 'req-de4eb784-8d4c-48b2-ac13-0056ce9b059f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.839353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbbba6ffd7864c80b3d3aea108547156', 'request_id': 'req-de4eb784-8d4c-48b2-ac13-0056ce9b059f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.847050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbbba6ffd7864c80b3d3aea108547156', 'request_id': 'req-de4eb784-8d4c-48b2-ac13-0056ce9b059f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.859265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] The token cache is being invalidate because project fbbba6ffd7864c80b3d3aea108547156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:33.859547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.860191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:33.861738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 941/1913] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:33 2022] DELETE /identity/v3/projects/fbbba6ffd7864c80b3d3aea108547156 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:33.865811 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7222d7-b6b3-497d-9ae7-181af7a21c6d service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:33.876286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7222d7-b6b3-497d-9ae7-181af7a21c6d service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:33.932808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:33.941807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:33.942642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:33.943111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:33.946943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] PATH_INFO: `/v3/users/07f4bea9d19d4777b836db94659edc18` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:33.946943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f4bea9d19d4777b836db94659edc18)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:33.948547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:33.948837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:33.950365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:33.969796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb784-8d4c-48b2-ac13-0056ce9b059f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:33.979152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:33.980542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:33.981001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:33.981450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] PATH_INFO: `/v3/projects/df0e421a1d13496ba5621c57426c7de7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:33.983454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0e421a1d13496ba5621c57426c7de7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:33.987352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:34.017876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f4bea9d19d4777b836db94659edc18'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.025112 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f4bea9d19d4777b836db94659edc18'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.034380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f4bea9d19d4777b836db94659edc18'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.035136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f4bea9d19d4777b836db94659edc18'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.035861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f4bea9d19d4777b836db94659edc18'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.057138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0e421a1d13496ba5621c57426c7de7', 'request_id': 'req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.057581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 973/1914] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:33 2022] DELETE /identity/v3/users/07f4bea9d19d4777b836db94659edc18 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:34.064061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0e421a1d13496ba5621c57426c7de7', 'request_id': 'req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.074041 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0e421a1d13496ba5621c57426c7de7', 'request_id': 'req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.089389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] The token cache is being invalidate because project df0e421a1d13496ba5621c57426c7de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:34.089681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.090548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.095303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 942/1915] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:33 2022] DELETE /identity/v3/projects/df0e421a1d13496ba5621c57426c7de7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:34.096294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:34.097762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:34.193334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555770cd-67ae-47ee-8a9b-141be9516166 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:34.200805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:34.202495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:34.202840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:34.203159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] PATH_INFO: `/v3/users/f145ade4546f43b5aa516ba702151c07` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:34.204798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f145ade4546f43b5aa516ba702151c07)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:34.207004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:34.280173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f145ade4546f43b5aa516ba702151c07'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.287461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f145ade4546f43b5aa516ba702151c07'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.296474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f145ade4546f43b5aa516ba702151c07'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.297320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f145ade4546f43b5aa516ba702151c07'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.298082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7d197cfd-178b-4351-b032-83f88523bcab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f145ade4546f43b5aa516ba702151c07'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.312934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:34.322034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 974/1916] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:34 2022] DELETE /identity/v3/users/f145ade4546f43b5aa516ba702151c07 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:34.505903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:34.508020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:34.614703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3e0d0-ebfc-4494-b5ab-f9596df229aa admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:34.676424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:34.685663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:34.686521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:34.686949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:34.687546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] PATH_INFO: `/v3/projects/1aa47b5ec5ce48ecaeb72e1c61fe50c7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:34.688971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa47b5ec5ce48ecaeb72e1c61fe50c7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:34.691293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:34.741995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa47b5ec5ce48ecaeb72e1c61fe50c7', 'request_id': 'req-bc7252d6-555b-4ae9-84f1-863e5b975f10'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.748214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa47b5ec5ce48ecaeb72e1c61fe50c7', 'request_id': 'req-bc7252d6-555b-4ae9-84f1-863e5b975f10'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.768465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa47b5ec5ce48ecaeb72e1c61fe50c7', 'request_id': 'req-bc7252d6-555b-4ae9-84f1-863e5b975f10'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.774628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] The token cache is being invalidate because project 1aa47b5ec5ce48ecaeb72e1c61fe50c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:34.774910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.775492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.777920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 943/1917] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:34 2022] DELETE /identity/v3/projects/1aa47b5ec5ce48ecaeb72e1c61fe50c7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:34.783315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5dc36-f1f7-4870-a861-55c4533a19c2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:34.783602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5dc36-f1f7-4870-a861-55c4533a19c2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:34.783879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5dc36-f1f7-4870-a861-55c4533a19c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:34.804505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e7d5dc36-f1f7-4870-a861-55c4533a19c2 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:34.805915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e7d5dc36-f1f7-4870-a861-55c4533a19c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:34.838106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:34.839883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:34.876353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 975/1918] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:34.885081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7301-a2f2-4f22-a362-49ea6b014fca None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:34.885782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7301-a2f2-4f22-a362-49ea6b014fca None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:34.886068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed7301-a2f2-4f22-a362-49ea6b014fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:34.906105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b5ed7301-a2f2-4f22-a362-49ea6b014fca None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:34.907332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b5ed7301-a2f2-4f22-a362-49ea6b014fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:34.916212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 976/1919] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:34.920352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed7301-a2f2-4f22-a362-49ea6b014fca None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:34.922339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7252d6-555b-4ae9-84f1-863e5b975f10 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:34.927320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:34.927898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:34.928208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:34.928501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] PATH_INFO: `/v3/projects/1233921319a84d188e5f32d2714c7a00` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:34.930210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed7301-a2f2-4f22-a362-49ea6b014fca None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:34.931136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1233921319a84d188e5f32d2714c7a00)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:34.933323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:34.934829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:34.935425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:34.935706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:34.935977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:34.936658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:34.936965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:34.938602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:34.954823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 977/1920] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:14:34.955529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:34.966848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39391aa-a028-40b2-85e5-fe505e4a83d9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:34.971598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:34.972184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:34.972482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:34.972757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:34.973372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:34.975288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:34.982052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1233921319a84d188e5f32d2714c7a00', 'request_id': 'req-e20678b4-7126-4add-b762-5e251e97b2e3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.988786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1233921319a84d188e5f32d2714c7a00', 'request_id': 'req-e20678b4-7126-4add-b762-5e251e97b2e3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:34.997846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1233921319a84d188e5f32d2714c7a00', 'request_id': 'req-e20678b4-7126-4add-b762-5e251e97b2e3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:35.008661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 978/1921] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:34 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:35.011365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] The token cache is being invalidate because project 1233921319a84d188e5f32d2714c7a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:35.011656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:35.012308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e20678b4-7126-4add-b762-5e251e97b2e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:35.012750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.013915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 944/1922] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:34 2022] DELETE /identity/v3/projects/1233921319a84d188e5f32d2714c7a00 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:35.014295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:35.018235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef600-7a44-408a-a92b-915e02a25f54 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.018514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef600-7a44-408a-a92b-915e02a25f54 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.018785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef600-7a44-408a-a92b-915e02a25f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.039064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-cc5ef600-7a44-408a-a92b-915e02a25f54 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:35.040248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cc5ef600-7a44-408a-a92b-915e02a25f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:35.051007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:35.079934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9da82f-d8c2-4b03-9dc5-c974e713c4f4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.085782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eff93-7386-45f5-8253-98dfe748ac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.086730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eff93-7386-45f5-8253-98dfe748ac89 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.087012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eff93-7386-45f5-8253-98dfe748ac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.087284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eff93-7386-45f5-8253-98dfe748ac89 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.088221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73eff93-7386-45f5-8253-98dfe748ac89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.090315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73eff93-7386-45f5-8253-98dfe748ac89 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.105612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 945/1923] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:35.115014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 979/1924] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:35.115515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e35d60-972b-4236-87ef-0e37034aaa99 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.116101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e35d60-972b-4236-87ef-0e37034aaa99 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.116314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e35d60-972b-4236-87ef-0e37034aaa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.121747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eff93-7386-45f5-8253-98dfe748ac89 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.132780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eff93-7386-45f5-8253-98dfe748ac89 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.137410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32423c6-d29b-4732-a0a7-53cb617e79d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.138029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32423c6-d29b-4732-a0a7-53cb617e79d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.138294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32423c6-d29b-4732-a0a7-53cb617e79d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.138475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a9e35d60-972b-4236-87ef-0e37034aaa99 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:35.138573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32423c6-d29b-4732-a0a7-53cb617e79d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.139332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32423c6-d29b-4732-a0a7-53cb617e79d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.139781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a9e35d60-972b-4236-87ef-0e37034aaa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:35.141053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32423c6-d29b-4732-a0a7-53cb617e79d9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.151889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 946/1925] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:35.152917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e35d60-972b-4236-87ef-0e37034aaa99 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.154260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 980/1926] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:35.157854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32423c6-d29b-4732-a0a7-53cb617e79d9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.163291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e35d60-972b-4236-87ef-0e37034aaa99 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.167860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.168101 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32423c6-d29b-4732-a0a7-53cb617e79d9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.168477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.168756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.169021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.169661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:35.169961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.171529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.172829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb0269-235a-49c3-b93e-d10733852af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.173490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb0269-235a-49c3-b93e-d10733852af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.173769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb0269-235a-49c3-b93e-d10733852af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.174047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb0269-235a-49c3-b93e-d10733852af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.174679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb0269-235a-49c3-b93e-d10733852af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.176878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb0269-235a-49c3-b93e-d10733852af6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.187774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 947/1927] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:35.191485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 981/1928] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:35.191606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.192390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb0269-235a-49c3-b93e-d10733852af6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.199593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a912d-3f9a-46aa-9e79-232f963a1fca admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.202618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb0269-235a-49c3-b93e-d10733852af6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.204257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e848d-08a0-4d5b-9850-9ae899995672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.204862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e848d-08a0-4d5b-9850-9ae899995672 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.205141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e848d-08a0-4d5b-9850-9ae899995672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.205408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e848d-08a0-4d5b-9850-9ae899995672 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.206020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e848d-08a0-4d5b-9850-9ae899995672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.207438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16695211-adbf-4b27-97e6-ca35d5e85bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.207880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e848d-08a0-4d5b-9850-9ae899995672 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.208064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16695211-adbf-4b27-97e6-ca35d5e85bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.208340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16695211-adbf-4b27-97e6-ca35d5e85bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.208645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16695211-adbf-4b27-97e6-ca35d5e85bb7 admin admin] PATH_INFO: `/v3/projects/54bc0e805465469ea3924d8117eea16e/users/ac2697dc5ee54a6084c48d8e596c7a82/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.224380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16695211-adbf-4b27-97e6-ca35d5e85bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54bc0e805465469ea3924d8117eea16e, user_id=ac2697dc5ee54a6084c48d8e596c7a82, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.227235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16695211-adbf-4b27-97e6-ca35d5e85bb7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.230745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:35.239531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 948/1929] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:35.240888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 982/1930] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:35 2022] PUT /identity/v3/projects/54bc0e805465469ea3924d8117eea16e/users/ac2697dc5ee54a6084c48d8e596c7a82/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:35.250020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e848d-08a0-4d5b-9850-9ae899995672 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.254653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32131f30-bb43-489a-8242-d5599f96e181 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.254939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32131f30-bb43-489a-8242-d5599f96e181 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.256205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32131f30-bb43-489a-8242-d5599f96e181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.286397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-32131f30-bb43-489a-8242-d5599f96e181 None None] MFA Rules not processed for user `ac2697dc5ee54a6084c48d8e596c7a82`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:35.287718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-32131f30-bb43-489a-8242-d5599f96e181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:35.334390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e848d-08a0-4d5b-9850-9ae899995672 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.346120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ef340-03a3-47d0-bda1-5e8e630dbe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.346744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef340-03a3-47d0-bda1-5e8e630dbe48 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.347028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef340-03a3-47d0-bda1-5e8e630dbe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.347391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef340-03a3-47d0-bda1-5e8e630dbe48 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.348025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ef340-03a3-47d0-bda1-5e8e630dbe48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.350502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ef340-03a3-47d0-bda1-5e8e630dbe48 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.376899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 949/1931] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:35.378224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ef340-03a3-47d0-bda1-5e8e630dbe48 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.390516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ef340-03a3-47d0-bda1-5e8e630dbe48 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.392190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 983/1932] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:35.396911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32131f30-bb43-489a-8242-d5599f96e181 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.397131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18280ac0-f3c4-45ad-9e4e-b4dc0f9c954e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.398744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18280ac0-f3c4-45ad-9e4e-b4dc0f9c954e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.399357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18280ac0-f3c4-45ad-9e4e-b4dc0f9c954e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.399780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18280ac0-f3c4-45ad-9e4e-b4dc0f9c954e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.400712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18280ac0-f3c4-45ad-9e4e-b4dc0f9c954e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.403699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18280ac0-f3c4-45ad-9e4e-b4dc0f9c954e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.409530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32131f30-bb43-489a-8242-d5599f96e181 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.415665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5cb78-7b7d-4e7b-ab45-0c654187b875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.415824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 950/1933] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:35.416561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5cb78-7b7d-4e7b-ab45-0c654187b875 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.416914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5cb78-7b7d-4e7b-ab45-0c654187b875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.417255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5cb78-7b7d-4e7b-ab45-0c654187b875 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.417956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5cb78-7b7d-4e7b-ab45-0c654187b875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.419381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18280ac0-f3c4-45ad-9e4e-b4dc0f9c954e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.420524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5cb78-7b7d-4e7b-ab45-0c654187b875 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.430833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18280ac0-f3c4-45ad-9e4e-b4dc0f9c954e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.436716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873be0c-deef-4cf7-9738-5247fa273a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.437399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873be0c-deef-4cf7-9738-5247fa273a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.437745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873be0c-deef-4cf7-9738-5247fa273a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.438086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873be0c-deef-4cf7-9738-5247fa273a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.438823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873be0c-deef-4cf7-9738-5247fa273a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.441102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873be0c-deef-4cf7-9738-5247fa273a00 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.452738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 984/1934] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:35.455139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 951/1935] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:35.458860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5cb78-7b7d-4e7b-ab45-0c654187b875 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.459009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873be0c-deef-4cf7-9738-5247fa273a00 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.470140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873be0c-deef-4cf7-9738-5247fa273a00 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.521001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:35.527678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5cb78-7b7d-4e7b-ab45-0c654187b875 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.532600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70f9f-b129-4929-960d-eecf2a69f681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.533211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70f9f-b129-4929-960d-eecf2a69f681 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.533496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70f9f-b129-4929-960d-eecf2a69f681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.533771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70f9f-b129-4929-960d-eecf2a69f681 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.534395 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed70f9f-b129-4929-960d-eecf2a69f681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.536326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed70f9f-b129-4929-960d-eecf2a69f681 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.540049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1c81b-186a-4cf6-b5eb-383ff18d2f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.540753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1c81b-186a-4cf6-b5eb-383ff18d2f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.541060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1c81b-186a-4cf6-b5eb-383ff18d2f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.541336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1c81b-186a-4cf6-b5eb-383ff18d2f27 admin admin] PATH_INFO: `/v3/projects/c355ef41e7974995b022c3f5a6e23295/users/c68d9c44293845b8910cabd9768db429/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.557079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1c81b-186a-4cf6-b5eb-383ff18d2f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c355ef41e7974995b022c3f5a6e23295, user_id=c68d9c44293845b8910cabd9768db429, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.562020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 985/1936] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/users => generated 306 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:35.562158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1c81b-186a-4cf6-b5eb-383ff18d2f27 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.563039 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70f9f-b129-4929-960d-eecf2a69f681 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.573907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70f9f-b129-4929-960d-eecf2a69f681 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.574640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 952/1937] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:35 2022] PUT /identity/v3/projects/c355ef41e7974995b022c3f5a6e23295/users/c68d9c44293845b8910cabd9768db429/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:35.586648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1fb8c-1063-48f1-a83d-0721dd1189e0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.587210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1fb8c-1063-48f1-a83d-0721dd1189e0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.587799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1fb8c-1063-48f1-a83d-0721dd1189e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.619426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-73f1fb8c-1063-48f1-a83d-0721dd1189e0 None None] MFA Rules not processed for user `c68d9c44293845b8910cabd9768db429`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:35.620815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-73f1fb8c-1063-48f1-a83d-0721dd1189e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:35.632847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:35.646131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5e189-48ab-41df-ad88-beb9c13975e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.646979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5e189-48ab-41df-ad88-beb9c13975e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.647382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5e189-48ab-41df-ad88-beb9c13975e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.648700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5e189-48ab-41df-ad88-beb9c13975e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.649627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5e189-48ab-41df-ad88-beb9c13975e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.651334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5e189-48ab-41df-ad88-beb9c13975e4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.662265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 986/1938] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:35.666401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5e189-48ab-41df-ad88-beb9c13975e4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.678015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5e189-48ab-41df-ad88-beb9c13975e4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.684035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cf2fa-e19c-4896-80a5-22eccf745300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.684819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf2fa-e19c-4896-80a5-22eccf745300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.684819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf2fa-e19c-4896-80a5-22eccf745300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.685009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf2fa-e19c-4896-80a5-22eccf745300 admin admin] PATH_INFO: `/v3/projects/2d0459a2919c4ba995ffc3977fe54f7c/users/49f6652af21e4f4daf8234ec4dcbad82/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.695563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 953/1939] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:35.703783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cf2fa-e19c-4896-80a5-22eccf745300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0459a2919c4ba995ffc3977fe54f7c, user_id=49f6652af21e4f4daf8234ec4dcbad82, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.706481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cf2fa-e19c-4896-80a5-22eccf745300 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.706765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1fb8c-1063-48f1-a83d-0721dd1189e0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.708889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-73f1fb8c-1063-48f1-a83d-0721dd1189e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:35.722851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 987/1940] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:35 2022] PUT /identity/v3/projects/2d0459a2919c4ba995ffc3977fe54f7c/users/49f6652af21e4f4daf8234ec4dcbad82/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:35.727156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cf2fa-e19c-4896-80a5-22eccf745300 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.798715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cf2fa-e19c-4896-80a5-22eccf745300 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.803431 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1fb8c-1063-48f1-a83d-0721dd1189e0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.803885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e4325-a10d-411c-949b-042d4735cda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.804705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e4325-a10d-411c-949b-042d4735cda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.804983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e4325-a10d-411c-949b-042d4735cda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.805307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e4325-a10d-411c-949b-042d4735cda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.805972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e4325-a10d-411c-949b-042d4735cda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.807570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e4325-a10d-411c-949b-042d4735cda3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.818009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 988/1941] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:35.822996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e4325-a10d-411c-949b-042d4735cda3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:35.837072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e4325-a10d-411c-949b-042d4735cda3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:35.842573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1856d3-454f-4754-be5b-973a724e8fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.843349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1856d3-454f-4754-be5b-973a724e8fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.843733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1856d3-454f-4754-be5b-973a724e8fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.844114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1856d3-454f-4754-be5b-973a724e8fe3 admin admin] PATH_INFO: `/v3/projects/2d0459a2919c4ba995ffc3977fe54f7c/users/49f6652af21e4f4daf8234ec4dcbad82/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.846207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1856d3-454f-4754-be5b-973a724e8fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0459a2919c4ba995ffc3977fe54f7c, user_id=49f6652af21e4f4daf8234ec4dcbad82, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.849061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1856d3-454f-4754-be5b-973a724e8fe3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.861083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 989/1942] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:35 2022] PUT /identity/v3/projects/2d0459a2919c4ba995ffc3977fe54f7c/users/49f6652af21e4f4daf8234ec4dcbad82/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:35.861960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a88817f3a4402290e9358826a63904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:35.862575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.862852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.863124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.867957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:35.871985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1bcd8-98ef-4d6a-a2c3-e33cf79ca9a4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:35.872502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1bcd8-98ef-4d6a-a2c3-e33cf79ca9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:35.873365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1bcd8-98ef-4d6a-a2c3-e33cf79ca9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:35.909337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-69d1bcd8-98ef-4d6a-a2c3-e33cf79ca9a4 None None] MFA Rules not processed for user `49f6652af21e4f4daf8234ec4dcbad82`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:35.910682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-69d1bcd8-98ef-4d6a-a2c3-e33cf79ca9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:35.930005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:35.931446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:35.940956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 954/1943] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:35.996038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 990/1944] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:35 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:36.007070 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:36.009421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:36.080293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129656bd-dc5c-495f-b731-e324812cd7e6 service glance] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:36.131176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319ebed-d2a6-4bf8-8057-b5877dcaeef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:36.131864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ebed-d2a6-4bf8-8057-b5877dcaeef4 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:36.132691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ebed-d2a6-4bf8-8057-b5877dcaeef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:36.132965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319ebed-d2a6-4bf8-8057-b5877dcaeef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:36.143822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319ebed-d2a6-4bf8-8057-b5877dcaeef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:36.145468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319ebed-d2a6-4bf8-8057-b5877dcaeef4 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:36.156947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 955/1945] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:36 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:36.257505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1bcd8-98ef-4d6a-a2c3-e33cf79ca9a4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:36.267377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1bcd8-98ef-4d6a-a2c3-e33cf79ca9a4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:36.271066 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08799cb-2230-4147-9e30-4c92c69bf2a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:36.271666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08799cb-2230-4147-9e30-4c92c69bf2a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:36.272399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08799cb-2230-4147-9e30-4c92c69bf2a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:36.272399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08799cb-2230-4147-9e30-4c92c69bf2a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:36.307885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:36.341185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08799cb-2230-4147-9e30-4c92c69bf2a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:36.342977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08799cb-2230-4147-9e30-4c92c69bf2a4 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:36.353893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 991/1946] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:39.826383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a7a33-b84f-4cb7-981d-9aae50a4531f None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:39.826383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a7a33-b84f-4cb7-981d-9aae50a4531f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:39.826383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a7a33-b84f-4cb7-981d-9aae50a4531f None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:39.827259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 956/1947] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:14:39.838043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6691-90f6-4e42-b2c7-2b980bab4cc8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:39.838723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6691-90f6-4e42-b2c7-2b980bab4cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:39.838723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf6691-90f6-4e42-b2c7-2b980bab4cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:39.881051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1ccf6691-90f6-4e42-b2c7-2b980bab4cc8 None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:39.882242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1ccf6691-90f6-4e42-b2c7-2b980bab4cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:39.974782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 992/1948] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:14:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:39.994106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a7a33-b84f-4cb7-981d-9aae50a4531f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:39.997365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b71a7a33-b84f-4cb7-981d-9aae50a4531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:40.021700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a7a33-b84f-4cb7-981d-9aae50a4531f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:40.031461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:40.032524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:40.033035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:40.033502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:40.048551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:40.052214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:40.071593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 957/1949] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:45.120077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf6691-90f6-4e42-b2c7-2b980bab4cc8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:45.211230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf6691-90f6-4e42-b2c7-2b980bab4cc8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:45.216376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:45.217065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:45.217358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:45.217674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] PATH_INFO: `/v3/users/ac2697dc5ee54a6084c48d8e596c7a82` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:45.221075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2697dc5ee54a6084c48d8e596c7a82)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:45.223206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:45.232249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:45.233707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:45.252923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd29b5-7ebf-4b5e-ae45-f5efc21ffa8e service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:45.258549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:45.259244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:45.259606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:45.260019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] PATH_INFO: `/v3/users/a002c6d8462f47929885473d594a77a9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:45.264892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a002c6d8462f47929885473d594a77a9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:45.267402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:45.332845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:45.340560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a002c6d8462f47929885473d594a77a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.340700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2697dc5ee54a6084c48d8e596c7a82'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.345170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a002c6d8462f47929885473d594a77a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.346589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2697dc5ee54a6084c48d8e596c7a82'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.353504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a002c6d8462f47929885473d594a77a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.354263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a002c6d8462f47929885473d594a77a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.354617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2697dc5ee54a6084c48d8e596c7a82'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.355133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-615357a5-09f2-430c-8a03-a34b09343b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a002c6d8462f47929885473d594a77a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.355288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2697dc5ee54a6084c48d8e596c7a82'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.355810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f076e495-6b72-4cf6-bfc6-448d44a47b5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2697dc5ee54a6084c48d8e596c7a82'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:45.357743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:45.374752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 958/1950] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:45 2022] DELETE /identity/v3/users/a002c6d8462f47929885473d594a77a9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:45.375584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 993/1951] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:45 2022] DELETE /identity/v3/users/ac2697dc5ee54a6084c48d8e596c7a82 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:45.379433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a537587-6d61-436b-84ee-0959773214fa None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:45.379719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497d7b0-6c4f-42a5-adbf-03984e27a8cc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:45.379913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a537587-6d61-436b-84ee-0959773214fa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:45.380007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497d7b0-6c4f-42a5-adbf-03984e27a8cc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:45.380273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497d7b0-6c4f-42a5-adbf-03984e27a8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:45.380610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a537587-6d61-436b-84ee-0959773214fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:45.401176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a497d7b0-6c4f-42a5-adbf-03984e27a8cc None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:45.402411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a497d7b0-6c4f-42a5-adbf-03984e27a8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:45.406812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8a537587-6d61-436b-84ee-0959773214fa None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:45.408098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8a537587-6d61-436b-84ee-0959773214fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:45.476899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 959/1952] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:45.486710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a537587-6d61-436b-84ee-0959773214fa None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:45.488956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8a537587-6d61-436b-84ee-0959773214fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:45.497078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 994/1953] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:45.508168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497d7b0-6c4f-42a5-adbf-03984e27a8cc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:45.509607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a497d7b0-6c4f-42a5-adbf-03984e27a8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:45.550596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a537587-6d61-436b-84ee-0959773214fa None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:45.555499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:45.556272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:45.556484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:45.556764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:45.567537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:45.569962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:45.570438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497d7b0-6c4f-42a5-adbf-03984e27a8cc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:45.575670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:45.576283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:45.576601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:45.576902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:45.581793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 960/1954] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:45.587284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:45.588975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:45.598908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 995/1955] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:46.075754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:46.078397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:46.111625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558ed9c-d846-4861-b0e4-02cfcf754200 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:46.122612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:46.123371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:46.123665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:46.123927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] PATH_INFO: `/v3/projects/54bc0e805465469ea3924d8117eea16e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:46.126336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54bc0e805465469ea3924d8117eea16e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:46.129333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:46.242781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:46.246105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:46.268942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54bc0e805465469ea3924d8117eea16e', 'request_id': 'req-fd8545d9-2dbf-42e1-877c-70d85a7ed782'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.281373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54bc0e805465469ea3924d8117eea16e', 'request_id': 'req-fd8545d9-2dbf-42e1-877c-70d85a7ed782'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.284426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691662c-3cc5-42d0-afbc-2707e77a78d2 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:46.287142 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:46.302636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54bc0e805465469ea3924d8117eea16e', 'request_id': 'req-fd8545d9-2dbf-42e1-877c-70d85a7ed782'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.364649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] The token cache is being invalidate because project 54bc0e805465469ea3924d8117eea16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:46.364649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.364649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.365259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 961/1956] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:46 2022] DELETE /identity/v3/projects/54bc0e805465469ea3924d8117eea16e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:46.371383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:46.377417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:46.417165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:46.418986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:46.419274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:46.421640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] PATH_INFO: `/v3/projects/9445777028a94a0386b4006db80049ce` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:46.423750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9445777028a94a0386b4006db80049ce)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:46.429367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:46.532872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:46.592057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8545d9-2dbf-42e1-877c-70d85a7ed782 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:46.597495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9445777028a94a0386b4006db80049ce', 'request_id': 'req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.605937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9445777028a94a0386b4006db80049ce', 'request_id': 'req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.635668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9445777028a94a0386b4006db80049ce', 'request_id': 'req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.652749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] The token cache is being invalidate because project 9445777028a94a0386b4006db80049ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:46.653180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.653927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f703e3fe-7f3b-42b0-b6c2-df3c1f22a3c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.665100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 996/1957] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:46 2022] DELETE /identity/v3/projects/9445777028a94a0386b4006db80049ce => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:46.670460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a54-322b-4c3d-994c-b9091769183d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:46.670839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a54-322b-4c3d-994c-b9091769183d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:46.671268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a54-322b-4c3d-994c-b9091769183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:46.719658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-96378a54-322b-4c3d-994c-b9091769183d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:46.722599 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-96378a54-322b-4c3d-994c-b9091769183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:46.787048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:46.788537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:46.789566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:46.790455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] PATH_INFO: `/v3/users/49f6652af21e4f4daf8234ec4dcbad82` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:46.796310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49f6652af21e4f4daf8234ec4dcbad82)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:46.800310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:46.840559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 997/1958] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:46.851545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fa366-a3b4-4984-bc66-e61fd892a806 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:46.852151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fa366-a3b4-4984-bc66-e61fd892a806 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:46.853563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fa366-a3b4-4984-bc66-e61fd892a806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:46.877055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e71fa366-a3b4-4984-bc66-e61fd892a806 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:46.878403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e71fa366-a3b4-4984-bc66-e61fd892a806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:46.886588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49f6652af21e4f4daf8234ec4dcbad82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.888484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 998/1959] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:46.892831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fa366-a3b4-4984-bc66-e61fd892a806 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:46.904726 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fa366-a3b4-4984-bc66-e61fd892a806 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:46.905255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49f6652af21e4f4daf8234ec4dcbad82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.913408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:46.915234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:46.915234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:46.915234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:46.916380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:46.916877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:46.919080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:46.932507 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49f6652af21e4f4daf8234ec4dcbad82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.933246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49f6652af21e4f4daf8234ec4dcbad82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.934056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49f6652af21e4f4daf8234ec4dcbad82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:46.948678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 999/1960] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:14:46.949569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:46.951671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:46.961670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 962/1961] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:46 2022] DELETE /identity/v3/users/49f6652af21e4f4daf8234ec4dcbad82 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:46.971452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8401b83-d259-41b6-9d4c-2a95049cda36 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.083118 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975188cc-7a8d-4699-9bdd-ac4f69599565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.083752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975188cc-7a8d-4699-9bdd-ac4f69599565 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.084029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975188cc-7a8d-4699-9bdd-ac4f69599565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.084316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975188cc-7a8d-4699-9bdd-ac4f69599565 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.085724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975188cc-7a8d-4699-9bdd-ac4f69599565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.087962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975188cc-7a8d-4699-9bdd-ac4f69599565 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.123783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1000/1962] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:46 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:47.127944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.141087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fece6-647a-450d-987e-7fa5c201a2d0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.155682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:47.220484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb79ae8-0cba-4bf3-9d99-0703cae6266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.220484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb79ae8-0cba-4bf3-9d99-0703cae6266d admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.220709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb79ae8-0cba-4bf3-9d99-0703cae6266d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.220946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb79ae8-0cba-4bf3-9d99-0703cae6266d admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.221588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb79ae8-0cba-4bf3-9d99-0703cae6266d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.223500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb79ae8-0cba-4bf3-9d99-0703cae6266d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.261614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 963/1963] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:47 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:47.262629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975188cc-7a8d-4699-9bdd-ac4f69599565 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.275370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975188cc-7a8d-4699-9bdd-ac4f69599565 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.280950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4817cc-468b-4358-b783-9724bccd7619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.281627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4817cc-468b-4358-b783-9724bccd7619 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.282000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4817cc-468b-4358-b783-9724bccd7619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.282341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4817cc-468b-4358-b783-9724bccd7619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.283047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4817cc-468b-4358-b783-9724bccd7619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.284804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4817cc-468b-4358-b783-9724bccd7619 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.300240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1001/1964] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:47.301469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb79ae8-0cba-4bf3-9d99-0703cae6266d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.314076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb79ae8-0cba-4bf3-9d99-0703cae6266d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.320170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097a814-395a-4d4f-b3de-9b0a5c9e739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.320767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097a814-395a-4d4f-b3de-9b0a5c9e739f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.321055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097a814-395a-4d4f-b3de-9b0a5c9e739f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.321681 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097a814-395a-4d4f-b3de-9b0a5c9e739f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.322352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097a814-395a-4d4f-b3de-9b0a5c9e739f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.324223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097a814-395a-4d4f-b3de-9b0a5c9e739f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.340873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 964/1965] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:47.342793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4817cc-468b-4358-b783-9724bccd7619 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.358574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4817cc-468b-4358-b783-9724bccd7619 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.372934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.374179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.375923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.376200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] PATH_INFO: `/v3/projects/7c2eaebf216d4ad5b7bf817ba9bbdef6/users/0d9fa00f83b74a2b85d5e6de22af4072/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.394740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:47.399231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2eaebf216d4ad5b7bf817ba9bbdef6, user_id=0d9fa00f83b74a2b85d5e6de22af4072, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.405392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.426673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1002/1966] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:47 2022] PUT /identity/v3/projects/7c2eaebf216d4ad5b7bf817ba9bbdef6/users/0d9fa00f83b74a2b85d5e6de22af4072/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:47.439080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56489238-3ad3-4e64-a8d7-ac0679f5dddd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.439358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56489238-3ad3-4e64-a8d7-ac0679f5dddd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.439635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56489238-3ad3-4e64-a8d7-ac0679f5dddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.446991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.449278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:47.472693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-56489238-3ad3-4e64-a8d7-ac0679f5dddd None None] MFA Rules not processed for user `0d9fa00f83b74a2b85d5e6de22af4072`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:47.474001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-56489238-3ad3-4e64-a8d7-ac0679f5dddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:47.581353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2877e-f98f-4a90-a123-7b16a3fc73c0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.590713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.590713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.590965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.591189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] PATH_INFO: `/v3/projects/2d0459a2919c4ba995ffc3977fe54f7c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.592640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d0459a2919c4ba995ffc3977fe54f7c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.595773 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.601371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 965/1967] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:47.605846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56489238-3ad3-4e64-a8d7-ac0679f5dddd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.620381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56489238-3ad3-4e64-a8d7-ac0679f5dddd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.629891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8479-3250-49b6-9eaf-5315107b8bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.630624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8479-3250-49b6-9eaf-5315107b8bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.630912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8479-3250-49b6-9eaf-5315107b8bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.631187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8479-3250-49b6-9eaf-5315107b8bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.633054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a8479-3250-49b6-9eaf-5315107b8bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.635426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a8479-3250-49b6-9eaf-5315107b8bfd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.668530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d0459a2919c4ba995ffc3977fe54f7c', 'request_id': 'req-e25c8df3-37a1-4464-bdf7-02117edaff51'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:47.686444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d0459a2919c4ba995ffc3977fe54f7c', 'request_id': 'req-e25c8df3-37a1-4464-bdf7-02117edaff51'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:47.698260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d0459a2919c4ba995ffc3977fe54f7c', 'request_id': 'req-e25c8df3-37a1-4464-bdf7-02117edaff51'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:47.698943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 966/1968] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:47 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:47.703018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8479-3250-49b6-9eaf-5315107b8bfd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.749018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] The token cache is being invalidate because project 2d0459a2919c4ba995ffc3977fe54f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:47.749524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:47.751747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e25c8df3-37a1-4464-bdf7-02117edaff51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:47.753910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1003/1969] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:47 2022] DELETE /identity/v3/projects/2d0459a2919c4ba995ffc3977fe54f7c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:47.760181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e523e3d-b159-4ee9-a69b-a518e31f6a8c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.760181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e523e3d-b159-4ee9-a69b-a518e31f6a8c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.760424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e523e3d-b159-4ee9-a69b-a518e31f6a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.782469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7e523e3d-b159-4ee9-a69b-a518e31f6a8c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:47.784387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7e523e3d-b159-4ee9-a69b-a518e31f6a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:47.815767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8479-3250-49b6-9eaf-5315107b8bfd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.822097 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.822737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.823020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.823340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.823958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.826053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.836238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:47.848793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 967/1970] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:47 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:47.854297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.855965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:47.869606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1004/1971] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:47.869828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9dcb1-e95f-4b6d-adc6-3062507c027b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.876110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08eb31-d689-4fe8-804d-5d73e673c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.876110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08eb31-d689-4fe8-804d-5d73e673c7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.876110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08eb31-d689-4fe8-804d-5d73e673c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.876110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08eb31-d689-4fe8-804d-5d73e673c7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.876706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08eb31-d689-4fe8-804d-5d73e673c7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.878045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d5a18-bb5f-464f-9b16-7588505baabc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.878343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d5a18-bb5f-464f-9b16-7588505baabc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.878561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08eb31-d689-4fe8-804d-5d73e673c7c1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.878662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d5a18-bb5f-464f-9b16-7588505baabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.891246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 968/1972] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:47.895612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08eb31-d689-4fe8-804d-5d73e673c7c1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.898873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f25d5a18-bb5f-464f-9b16-7588505baabc None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:47.900089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f25d5a18-bb5f-464f-9b16-7588505baabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:47.908155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1005/1973] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:47.908616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08eb31-d689-4fe8-804d-5d73e673c7c1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.912288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d5a18-bb5f-464f-9b16-7588505baabc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.916335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbe70d-6d44-4622-8f00-198f8ad04c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.916988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe70d-6d44-4622-8f00-198f8ad04c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.917270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe70d-6d44-4622-8f00-198f8ad04c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.917755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbe70d-6d44-4622-8f00-198f8ad04c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.918428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbe70d-6d44-4622-8f00-198f8ad04c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.920759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbe70d-6d44-4622-8f00-198f8ad04c7d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.923065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d5a18-bb5f-464f-9b16-7588505baabc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.930631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.931585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.932040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.933487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.934421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:47.934938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.936759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.943487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 969/1974] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:47.947998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:47.948204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbe70d-6d44-4622-8f00-198f8ad04c7d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.959297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1006/1975] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:14:47.961025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:47.962647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbe70d-6d44-4622-8f00-198f8ad04c7d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.967512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dc533-f6ca-4c0e-956f-15d734e96dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.969111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc533-f6ca-4c0e-956f-15d734e96dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.969384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc533-f6ca-4c0e-956f-15d734e96dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.969679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc533-f6ca-4c0e-956f-15d734e96dfc admin admin] PATH_INFO: `/v3/projects/ca313d2530f042459443481d3e9b2c56/users/1beb76e3d7d44af3941176d840714aa6/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.974530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029ef8a-6935-43b7-b03d-0bedfdc55777 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:47.980673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c7b17-2894-4fc8-9dcf-d58451d890f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:47.981285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7b17-2894-4fc8-9dcf-d58451d890f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:47.981560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7b17-2894-4fc8-9dcf-d58451d890f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:47.981833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7b17-2894-4fc8-9dcf-d58451d890f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:47.982454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c7b17-2894-4fc8-9dcf-d58451d890f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.984347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c7b17-2894-4fc8-9dcf-d58451d890f7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:47.985842 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dc533-f6ca-4c0e-956f-15d734e96dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca313d2530f042459443481d3e9b2c56, user_id=1beb76e3d7d44af3941176d840714aa6, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:47.988410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dc533-f6ca-4c0e-956f-15d734e96dfc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.005742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 970/1976] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:47 2022] PUT /identity/v3/projects/ca313d2530f042459443481d3e9b2c56/users/1beb76e3d7d44af3941176d840714aa6/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:48.016725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdae08-67c2-426a-98d3-006cac7841d6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.017068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdae08-67c2-426a-98d3-006cac7841d6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.017250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1007/1977] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:47 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:48.017410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdae08-67c2-426a-98d3-006cac7841d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.020864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c7b17-2894-4fc8-9dcf-d58451d890f7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.047181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f1bdae08-67c2-426a-98d3-006cac7841d6 None None] MFA Rules not processed for user `1beb76e3d7d44af3941176d840714aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:48.048760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f1bdae08-67c2-426a-98d3-006cac7841d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:48.131051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c7b17-2894-4fc8-9dcf-d58451d890f7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.133665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 971/1978] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:48.137150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f167-92a8-4308-894f-8b03f04f3362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.137787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f167-92a8-4308-894f-8b03f04f3362 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.138066 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f167-92a8-4308-894f-8b03f04f3362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.138184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdae08-67c2-426a-98d3-006cac7841d6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.138340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f167-92a8-4308-894f-8b03f04f3362 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.138965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5f167-92a8-4308-894f-8b03f04f3362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.140938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5f167-92a8-4308-894f-8b03f04f3362 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.149486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdae08-67c2-426a-98d3-006cac7841d6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.154141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db78cdf-d130-4d5e-9237-626bfc944bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.154814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db78cdf-d130-4d5e-9237-626bfc944bba admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.155258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db78cdf-d130-4d5e-9237-626bfc944bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.155757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db78cdf-d130-4d5e-9237-626bfc944bba admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.156475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db78cdf-d130-4d5e-9237-626bfc944bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.159098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db78cdf-d130-4d5e-9237-626bfc944bba admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.165529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1008/1979] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:48 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:48.169981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f167-92a8-4308-894f-8b03f04f3362 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.181417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f167-92a8-4308-894f-8b03f04f3362 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.186585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20095f8e-ef71-486f-9754-e18a92aad92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.187227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095f8e-ef71-486f-9754-e18a92aad92a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.187528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095f8e-ef71-486f-9754-e18a92aad92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.187771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095f8e-ef71-486f-9754-e18a92aad92a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.188544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20095f8e-ef71-486f-9754-e18a92aad92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.190424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20095f8e-ef71-486f-9754-e18a92aad92a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.195418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 972/1980] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:48 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:48.196634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db78cdf-d130-4d5e-9237-626bfc944bba admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.211410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1009/1981] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:48.213047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20095f8e-ef71-486f-9754-e18a92aad92a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.223153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20095f8e-ef71-486f-9754-e18a92aad92a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.265722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db78cdf-d130-4d5e-9237-626bfc944bba admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.270690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a816337-0f84-4ee9-9d01-d2e3e6f29d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.271296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a816337-0f84-4ee9-9d01-d2e3e6f29d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.271583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a816337-0f84-4ee9-9d01-d2e3e6f29d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.272649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a816337-0f84-4ee9-9d01-d2e3e6f29d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.273460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a816337-0f84-4ee9-9d01-d2e3e6f29d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.275375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a816337-0f84-4ee9-9d01-d2e3e6f29d66 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.281266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11265e-d286-46f0-9881-343279a6e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.281917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11265e-d286-46f0-9881-343279a6e722 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.282174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11265e-d286-46f0-9881-343279a6e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.282443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11265e-d286-46f0-9881-343279a6e722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.283079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11265e-d286-46f0-9881-343279a6e722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.284693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11265e-d286-46f0-9881-343279a6e722 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.298583 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1010/1982] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:48.298763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 973/1983] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:48 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:14:48.299524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11265e-d286-46f0-9881-343279a6e722 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.301194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a816337-0f84-4ee9-9d01-d2e3e6f29d66 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.306303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:48.309644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11265e-d286-46f0-9881-343279a6e722 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.312144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a816337-0f84-4ee9-9d01-d2e3e6f29d66 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.314327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5953a8c-c03b-4f06-a933-d65ba644dd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.314967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5953a8c-c03b-4f06-a933-d65ba644dd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.315245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5953a8c-c03b-4f06-a933-d65ba644dd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.315564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5953a8c-c03b-4f06-a933-d65ba644dd41 admin admin] PATH_INFO: `/v3/projects/9cdb248063644b56b03751405262503b/users/7eb3b336df44491f8a973a9b2a53d1d2/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.317400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdc311-df25-418a-98f2-a8f708532efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.318174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdc311-df25-418a-98f2-a8f708532efd admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.319888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdc311-df25-418a-98f2-a8f708532efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.320181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdc311-df25-418a-98f2-a8f708532efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.320910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdc311-df25-418a-98f2-a8f708532efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.322595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdc311-df25-418a-98f2-a8f708532efd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.331912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5953a8c-c03b-4f06-a933-d65ba644dd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cdb248063644b56b03751405262503b, user_id=7eb3b336df44491f8a973a9b2a53d1d2, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.338968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 974/1984] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:48.339133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5953a8c-c03b-4f06-a933-d65ba644dd41 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.341092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdc311-df25-418a-98f2-a8f708532efd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.352336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdc311-df25-418a-98f2-a8f708532efd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.352836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1011/1985] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:48 2022] PUT /identity/v3/projects/9cdb248063644b56b03751405262503b/users/7eb3b336df44491f8a973a9b2a53d1d2/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:48.364216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89351d98-2070-4c98-b3e1-151bea59afe2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.364524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89351d98-2070-4c98-b3e1-151bea59afe2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.364805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89351d98-2070-4c98-b3e1-151bea59afe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.394837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-89351d98-2070-4c98-b3e1-151bea59afe2 None None] MFA Rules not processed for user `7eb3b336df44491f8a973a9b2a53d1d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:48.395225 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-89351d98-2070-4c98-b3e1-151bea59afe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:48.410234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:48.416278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1fcd-c794-4af6-9121-ff3a98b8f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.416985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1fcd-c794-4af6-9121-ff3a98b8f047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.417265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1fcd-c794-4af6-9121-ff3a98b8f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.417641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb1fcd-c794-4af6-9121-ff3a98b8f047 admin admin] PATH_INFO: `/v3/projects/97c45cdec2174b5da79430009b67cc84/users/695c136f22264cbeb6b48a80be400e1a/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.434737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb1fcd-c794-4af6-9121-ff3a98b8f047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c45cdec2174b5da79430009b67cc84, user_id=695c136f22264cbeb6b48a80be400e1a, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.437638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb1fcd-c794-4af6-9121-ff3a98b8f047 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.452904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 975/1986] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:48 2022] PUT /identity/v3/projects/97c45cdec2174b5da79430009b67cc84/users/695c136f22264cbeb6b48a80be400e1a/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:48.457025 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1fcd-c794-4af6-9121-ff3a98b8f047 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.488868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1012/1987] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:48.501051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89351d98-2070-4c98-b3e1-151bea59afe2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.502537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-89351d98-2070-4c98-b3e1-151bea59afe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:48.543820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb1fcd-c794-4af6-9121-ff3a98b8f047 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.548641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a732ec-206e-4f7f-a5a9-54c44e5d63c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.549350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a732ec-206e-4f7f-a5a9-54c44e5d63c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.549964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a732ec-206e-4f7f-a5a9-54c44e5d63c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.550303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a732ec-206e-4f7f-a5a9-54c44e5d63c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.551023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a732ec-206e-4f7f-a5a9-54c44e5d63c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.552799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a732ec-206e-4f7f-a5a9-54c44e5d63c6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.563489 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 976/1988] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:14:48.567651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a732ec-206e-4f7f-a5a9-54c44e5d63c6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.571269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89351d98-2070-4c98-b3e1-151bea59afe2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.578394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a732ec-206e-4f7f-a5a9-54c44e5d63c6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.583206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fae8f8-aa01-41ff-bbab-8c745be39b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.583875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae8f8-aa01-41ff-bbab-8c745be39b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.584240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae8f8-aa01-41ff-bbab-8c745be39b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.585217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae8f8-aa01-41ff-bbab-8c745be39b59 admin admin] PATH_INFO: `/v3/projects/97c45cdec2174b5da79430009b67cc84/users/695c136f22264cbeb6b48a80be400e1a/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.587330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fae8f8-aa01-41ff-bbab-8c745be39b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c45cdec2174b5da79430009b67cc84, user_id=695c136f22264cbeb6b48a80be400e1a, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.590020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fae8f8-aa01-41ff-bbab-8c745be39b59 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.601720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 977/1989] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:48 2022] PUT /identity/v3/projects/97c45cdec2174b5da79430009b67cc84/users/695c136f22264cbeb6b48a80be400e1a/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:48.613166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34faac-315c-49e5-90af-9326670f7e67 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.613380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34faac-315c-49e5-90af-9326670f7e67 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.613679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34faac-315c-49e5-90af-9326670f7e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.635754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.636384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.636705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.637133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.646558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2c34faac-315c-49e5-90af-9326670f7e67 None None] MFA Rules not processed for user `695c136f22264cbeb6b48a80be400e1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:48.648170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2c34faac-315c-49e5-90af-9326670f7e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:48.693567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:48.697248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.698737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.707952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1013/1990] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:14:48.720819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 978/1991] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:48.733343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:48.734754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:48.746160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96535d61-43a3-4a39-acc1-b80e0b474e8b service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:48.783988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:48.826321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:48.827249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:48.827635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:48.828025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:48.830046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:48.922410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:48.923878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:48.943400 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1014/1992] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:53.705078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34faac-315c-49e5-90af-9326670f7e67 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:53.719237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34faac-315c-49e5-90af-9326670f7e67 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:53.725582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:53.726337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:53.728038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:53.728526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:53.840630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:53.842414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:53.853615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 979/1993] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:54.734808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:54.743486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1dfe0f-d1a3-4376-b232-12bb92785969 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:54.748568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b894d3-1570-41ce-95ed-4de6545fc9f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:54.749335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b894d3-1570-41ce-95ed-4de6545fc9f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:54.749652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b894d3-1570-41ce-95ed-4de6545fc9f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:54.750450 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b894d3-1570-41ce-95ed-4de6545fc9f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:54.761867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b894d3-1570-41ce-95ed-4de6545fc9f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:54.763316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b894d3-1570-41ce-95ed-4de6545fc9f6 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:54.774281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1015/1994] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:55.832356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:55.833829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:55.900366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51761c34-783c-4e3b-a716-4ec6419cd39d service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:55.906911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:55.907632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:55.907972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:55.908309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] PATH_INFO: `/v3/users/1bb6b95759d642fe9dbc24793089a70f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:55.909571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb6b95759d642fe9dbc24793089a70f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:55.911597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:55.990190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb6b95759d642fe9dbc24793089a70f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:55.998404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb6b95759d642fe9dbc24793089a70f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.023715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb6b95759d642fe9dbc24793089a70f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.025792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb6b95759d642fe9dbc24793089a70f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.026571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb6b95759d642fe9dbc24793089a70f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.044195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:56.052598 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 980/1995] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:55 2022] DELETE /identity/v3/users/1bb6b95759d642fe9dbc24793089a70f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:56.054591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ebbf8-87b0-40ef-96f0-211cf9dc6594 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:56.055712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ebbf8-87b0-40ef-96f0-211cf9dc6594 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:56.056031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ebbf8-87b0-40ef-96f0-211cf9dc6594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:56.077850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d32ebbf8-87b0-40ef-96f0-211cf9dc6594 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:56.079078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d32ebbf8-87b0-40ef-96f0-211cf9dc6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:56.151229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1016/1996] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:56.164922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:56.167382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:56.256082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438128c4-a93a-4ee9-9f66-402db9eb5d5a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:56.260655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf382b1-dc70-401f-865d-54a1ed64008b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:56.261630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf382b1-dc70-401f-865d-54a1ed64008b service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:56.261902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf382b1-dc70-401f-865d-54a1ed64008b service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:56.262168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf382b1-dc70-401f-865d-54a1ed64008b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:56.276893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf382b1-dc70-401f-865d-54a1ed64008b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:56.278455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf382b1-dc70-401f-865d-54a1ed64008b service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:56.297113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 981/1997] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:56.755305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ebbf8-87b0-40ef-96f0-211cf9dc6594 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:56.757162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d32ebbf8-87b0-40ef-96f0-211cf9dc6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:56.769520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ebbf8-87b0-40ef-96f0-211cf9dc6594 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:56.774571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:56.775227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:56.775503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:56.775790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] PATH_INFO: `/v3/projects/1a0b9850f88142be9093b65265fbd8f4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:56.776991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0b9850f88142be9093b65265fbd8f4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:56.778951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:56.841546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0b9850f88142be9093b65265fbd8f4', 'request_id': 'req-cb1d9961-3588-427b-8a10-b443025ef7ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.846933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0b9850f88142be9093b65265fbd8f4', 'request_id': 'req-cb1d9961-3588-427b-8a10-b443025ef7ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.851439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:56.856957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0b9850f88142be9093b65265fbd8f4', 'request_id': 'req-cb1d9961-3588-427b-8a10-b443025ef7ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.871096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] The token cache is being invalidate because project 1a0b9850f88142be9093b65265fbd8f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:56.871391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.872078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:56.878474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1017/1998] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:56 2022] DELETE /identity/v3/projects/1a0b9850f88142be9093b65265fbd8f4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:56.882595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf382b1-dc70-401f-865d-54a1ed64008b service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:56.896073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf382b1-dc70-401f-865d-54a1ed64008b service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:56.965154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:56.965798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:56.966075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:56.966351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] PATH_INFO: `/v3/users/2534915a0ec9465d8abb4aea9a644958` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:56.967472 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2534915a0ec9465d8abb4aea9a644958)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:56.969461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:57.033212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2534915a0ec9465d8abb4aea9a644958'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.042008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2534915a0ec9465d8abb4aea9a644958'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.051426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2534915a0ec9465d8abb4aea9a644958'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.052379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2534915a0ec9465d8abb4aea9a644958'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.053304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-06bf5e73-d15e-417a-9cf0-8d26cda251e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2534915a0ec9465d8abb4aea9a644958'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.071196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 982/1999] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:56 2022] DELETE /identity/v3/users/2534915a0ec9465d8abb4aea9a644958 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:57.356402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:57.358212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:57.380895 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d9961-3588-427b-8a10-b443025ef7ed admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:57.469961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:57.470792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:57.471236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:57.471508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] PATH_INFO: `/v3/projects/4788c8cb0e304226a8ab2b38cec17219` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:57.472665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4788c8cb0e304226a8ab2b38cec17219)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:57.474589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:57.542298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4788c8cb0e304226a8ab2b38cec17219', 'request_id': 'req-b92bb4d6-8204-49c6-8119-a64a2da274d9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.548513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4788c8cb0e304226a8ab2b38cec17219', 'request_id': 'req-b92bb4d6-8204-49c6-8119-a64a2da274d9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.558196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4788c8cb0e304226a8ab2b38cec17219', 'request_id': 'req-b92bb4d6-8204-49c6-8119-a64a2da274d9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.575279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] The token cache is being invalidate because project 4788c8cb0e304226a8ab2b38cec17219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:14:57.575781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.576624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b92bb4d6-8204-49c6-8119-a64a2da274d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:57.578166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1018/2000] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:14:57 2022] DELETE /identity/v3/projects/4788c8cb0e304226a8ab2b38cec17219 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:57.583135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67748152-9c93-47b7-965c-09c72a489375 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:57.583419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67748152-9c93-47b7-965c-09c72a489375 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:57.583721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67748152-9c93-47b7-965c-09c72a489375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:57.609254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-67748152-9c93-47b7-965c-09c72a489375 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:57.610481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-67748152-9c93-47b7-965c-09c72a489375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:57.629158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:57.688515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 983/2001] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:14:57.692004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ad8e4-7f53-49a9-8fb9-c8781f32b9be None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:57.692361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ad8e4-7f53-49a9-8fb9-c8781f32b9be None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:57.692736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ad8e4-7f53-49a9-8fb9-c8781f32b9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:57.716167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-219ad8e4-7f53-49a9-8fb9-c8781f32b9be None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:57.717525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-219ad8e4-7f53-49a9-8fb9-c8781f32b9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:57.726822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1019/2002] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:57.731164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67748152-9c93-47b7-965c-09c72a489375 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:57.742480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67748152-9c93-47b7-965c-09c72a489375 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:57.754236 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:57.755133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:57.755416 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:57.755693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:57.756620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:14:57.756938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:57.758568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:57.770892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 984/2003] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:14:57.775900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ad8e4-7f53-49a9-8fb9-c8781f32b9be None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:57.785376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ad8e4-7f53-49a9-8fb9-c8781f32b9be None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:57.791059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac80472-231c-4a3d-a013-b535e728fcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:57.791670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80472-231c-4a3d-a013-b535e728fcd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:57.791950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80472-231c-4a3d-a013-b535e728fcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:57.792222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80472-231c-4a3d-a013-b535e728fcd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:57.792902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac80472-231c-4a3d-a013-b535e728fcd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:57.794792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac80472-231c-4a3d-a013-b535e728fcd8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:57.823576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1020/2004] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:57 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:57.827880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:57.839206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d1849-d892-4fca-97ae-808dcdda4e01 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:57.899790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94569d-d925-434d-b86a-fa3882aac4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:57.900616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94569d-d925-434d-b86a-fa3882aac4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:57.901000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94569d-d925-434d-b86a-fa3882aac4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:57.901512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94569d-d925-434d-b86a-fa3882aac4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:57.902153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94569d-d925-434d-b86a-fa3882aac4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:57.904120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94569d-d925-434d-b86a-fa3882aac4a3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:57.930131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 985/2005] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:57 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:57.934253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac80472-231c-4a3d-a013-b535e728fcd8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:57.945937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac80472-231c-4a3d-a013-b535e728fcd8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:57.950585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df03c5a-e938-4a3f-82f6-a3042c62855b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:57.951188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df03c5a-e938-4a3f-82f6-a3042c62855b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:57.951463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df03c5a-e938-4a3f-82f6-a3042c62855b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:57.951735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df03c5a-e938-4a3f-82f6-a3042c62855b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:57.952369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df03c5a-e938-4a3f-82f6-a3042c62855b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:57.953983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df03c5a-e938-4a3f-82f6-a3042c62855b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:57.964543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1021/2006] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:57.968686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94569d-d925-434d-b86a-fa3882aac4a3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:57.979971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94569d-d925-434d-b86a-fa3882aac4a3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:57.984976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39af5c8-2174-424e-ac4c-ebc74f6730a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:57.985585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af5c8-2174-424e-ac4c-ebc74f6730a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:57.985879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af5c8-2174-424e-ac4c-ebc74f6730a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:57.986567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af5c8-2174-424e-ac4c-ebc74f6730a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:57.987500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39af5c8-2174-424e-ac4c-ebc74f6730a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:57.989123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39af5c8-2174-424e-ac4c-ebc74f6730a8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.001029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 986/2007] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:58.005433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df03c5a-e938-4a3f-82f6-a3042c62855b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:58.016160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df03c5a-e938-4a3f-82f6-a3042c62855b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:58.021019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0b174-1a6a-4d89-972c-c488bb032039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:58.022255 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0b174-1a6a-4d89-972c-c488bb032039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.022592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0b174-1a6a-4d89-972c-c488bb032039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.022928 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0b174-1a6a-4d89-972c-c488bb032039 admin admin] PATH_INFO: `/v3/projects/bdcb2a05bd374645bc9393ab1061a20e/users/2d11f2ede1204e81b02567334881801f/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.041840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0b174-1a6a-4d89-972c-c488bb032039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcb2a05bd374645bc9393ab1061a20e, user_id=2d11f2ede1204e81b02567334881801f, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:58.044923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0b174-1a6a-4d89-972c-c488bb032039 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.060591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1022/2008] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:58 2022] PUT /identity/v3/projects/bdcb2a05bd374645bc9393ab1061a20e/users/2d11f2ede1204e81b02567334881801f/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:58.071453 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb1411-2e6c-4e07-85fd-6f56c818971c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.071799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb1411-2e6c-4e07-85fd-6f56c818971c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.072166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb1411-2e6c-4e07-85fd-6f56c818971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.103062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-12cb1411-2e6c-4e07-85fd-6f56c818971c None None] MFA Rules not processed for user `2d11f2ede1204e81b02567334881801f`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:58.104303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-12cb1411-2e6c-4e07-85fd-6f56c818971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:58.177415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 987/2009] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:58.182934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0b174-1a6a-4d89-972c-c488bb032039 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:58.211879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:58.261120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0b174-1a6a-4d89-972c-c488bb032039 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:58.265724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e78ac-cd18-42e7-a108-93ca6f8d9027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:58.266332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e78ac-cd18-42e7-a108-93ca6f8d9027 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.266607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e78ac-cd18-42e7-a108-93ca6f8d9027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.266878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e78ac-cd18-42e7-a108-93ca6f8d9027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.267510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e78ac-cd18-42e7-a108-93ca6f8d9027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:58.269436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e78ac-cd18-42e7-a108-93ca6f8d9027 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.300847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1023/2010] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:14:58 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:58.301863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb1411-2e6c-4e07-85fd-6f56c818971c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:58.322739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb1411-2e6c-4e07-85fd-6f56c818971c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:58.337628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:58.386808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efa43c-87d9-49af-9fae-fe66a9cc65cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:58.387436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efa43c-87d9-49af-9fae-fe66a9cc65cd admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.387712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efa43c-87d9-49af-9fae-fe66a9cc65cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.387983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efa43c-87d9-49af-9fae-fe66a9cc65cd admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.388682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efa43c-87d9-49af-9fae-fe66a9cc65cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:58.391172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efa43c-87d9-49af-9fae-fe66a9cc65cd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.413698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 988/2011] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:14:58 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:14:58.417629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e78ac-cd18-42e7-a108-93ca6f8d9027 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:58.438671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e78ac-cd18-42e7-a108-93ca6f8d9027 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:58.443469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1547-0b4b-4a09-8398-ca6c66f82d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:58.444924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1547-0b4b-4a09-8398-ca6c66f82d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.445491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1547-0b4b-4a09-8398-ca6c66f82d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.445658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1547-0b4b-4a09-8398-ca6c66f82d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.446375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb1547-0b4b-4a09-8398-ca6c66f82d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:58.448379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb1547-0b4b-4a09-8398-ca6c66f82d29 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.459471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1024/2012] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:58.464605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efa43c-87d9-49af-9fae-fe66a9cc65cd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:58.479181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efa43c-87d9-49af-9fae-fe66a9cc65cd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:58.484030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ab092-b26a-4312-95e4-01b0c7df423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:58.485049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ab092-b26a-4312-95e4-01b0c7df423d admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.485434 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ab092-b26a-4312-95e4-01b0c7df423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.485716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ab092-b26a-4312-95e4-01b0c7df423d admin admin] PATH_INFO: `/v3/projects/e8121ce17e244a7697059afa1ef42a86/users/f64fd04b4d76461e80efbcd3e41e1a13/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.502394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ab092-b26a-4312-95e4-01b0c7df423d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8121ce17e244a7697059afa1ef42a86, user_id=f64fd04b4d76461e80efbcd3e41e1a13, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:58.505260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ab092-b26a-4312-95e4-01b0c7df423d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.519709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 989/2013] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:58 2022] PUT /identity/v3/projects/e8121ce17e244a7697059afa1ef42a86/users/f64fd04b4d76461e80efbcd3e41e1a13/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:58.523790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1547-0b4b-4a09-8398-ca6c66f82d29 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:58.533885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1547-0b4b-4a09-8398-ca6c66f82d29 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:58.590784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec21540-5f22-4bf1-ae6a-c76f6c82491a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:58.591340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec21540-5f22-4bf1-ae6a-c76f6c82491a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.591609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec21540-5f22-4bf1-ae6a-c76f6c82491a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.591877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec21540-5f22-4bf1-ae6a-c76f6c82491a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.592695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec21540-5f22-4bf1-ae6a-c76f6c82491a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:58.594500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec21540-5f22-4bf1-ae6a-c76f6c82491a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.609563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1025/2014] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:14:58.610664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ab092-b26a-4312-95e4-01b0c7df423d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:58.623704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ab092-b26a-4312-95e4-01b0c7df423d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:58.628216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:58.628901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.629175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.629441 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] PATH_INFO: `/v3/projects/e8121ce17e244a7697059afa1ef42a86/users/f64fd04b4d76461e80efbcd3e41e1a13/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.631990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8121ce17e244a7697059afa1ef42a86, user_id=f64fd04b4d76461e80efbcd3e41e1a13, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:58.635754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.652473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 990/2015] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:14:58 2022] PUT /identity/v3/projects/e8121ce17e244a7697059afa1ef42a86/users/f64fd04b4d76461e80efbcd3e41e1a13/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:14:58.663864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094dc61-f1b6-48eb-80f5-c08189ea9021 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.664148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094dc61-f1b6-48eb-80f5-c08189ea9021 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.664488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094dc61-f1b6-48eb-80f5-c08189ea9021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.698123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e094dc61-f1b6-48eb-80f5-c08189ea9021 None None] MFA Rules not processed for user `f64fd04b4d76461e80efbcd3e41e1a13`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:58.699852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e094dc61-f1b6-48eb-80f5-c08189ea9021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:58.792309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1026/2016] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:14:58.805432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:58.807262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:58.888613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07f51-fa40-4cb7-9fa4-3bbd4a3dc55a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:58.892989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:58.893701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:58.894056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:58.894398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:58.906392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:58.908549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:58.934549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 991/2017] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:59.030980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094dc61-f1b6-48eb-80f5-c08189ea9021 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:59.043978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094dc61-f1b6-48eb-80f5-c08189ea9021 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:59.055242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b74f54-d877-4d98-8d7d-fc61bc9ca7bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:59.057124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74f54-d877-4d98-8d7d-fc61bc9ca7bc service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:59.057660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74f54-d877-4d98-8d7d-fc61bc9ca7bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:59.058022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74f54-d877-4d98-8d7d-fc61bc9ca7bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:59.087917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:59.168868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b74f54-d877-4d98-8d7d-fc61bc9ca7bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:59.170998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b74f54-d877-4d98-8d7d-fc61bc9ca7bc service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:59.188285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1027/2018] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:14:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:14:59.670460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:14:59.672398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:59.757916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6638d2a-dc76-4033-b25c-3a79ccf384a8 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:14:59.764638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:14:59.765268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:59.765534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:59.766315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] PATH_INFO: `/v3/users/7eb3b336df44491f8a973a9b2a53d1d2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:59.766934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eb3b336df44491f8a973a9b2a53d1d2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:14:59.768893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:14:59.798124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:14:59.845093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eb3b336df44491f8a973a9b2a53d1d2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:59.853981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eb3b336df44491f8a973a9b2a53d1d2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:59.862491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eb3b336df44491f8a973a9b2a53d1d2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:59.863465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eb3b336df44491f8a973a9b2a53d1d2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:59.864173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eb3b336df44491f8a973a9b2a53d1d2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:14:59.888626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 992/2019] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:14:59 2022] DELETE /identity/v3/users/7eb3b336df44491f8a973a9b2a53d1d2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:14:59.893318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a034-664e-407b-9537-4255e8f59ad8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:14:59.893937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a034-664e-407b-9537-4255e8f59ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:14:59.894105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a034-664e-407b-9537-4255e8f59ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:14:59.919878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5085a034-664e-407b-9537-4255e8f59ad8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:14:59.921130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5085a034-664e-407b-9537-4255e8f59ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:14:59.997389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1028/2020] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:00.008737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:00.010733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:00.071094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c255b-b4bc-42ec-99d2-cccaaf5b0ecf admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:00.074918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fd168-84a5-49f2-96e0-14d355a962f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:00.075536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd168-84a5-49f2-96e0-14d355a962f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.075820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd168-84a5-49f2-96e0-14d355a962f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.076128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fd168-84a5-49f2-96e0-14d355a962f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.086524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fd168-84a5-49f2-96e0-14d355a962f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:00.087951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fd168-84a5-49f2-96e0-14d355a962f2 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:00.098057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 993/2021] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:00.381319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085a034-664e-407b-9537-4255e8f59ad8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:00.383352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5085a034-664e-407b-9537-4255e8f59ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:00.398315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085a034-664e-407b-9537-4255e8f59ad8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:00.404073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:00.404884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.405166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.405440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] PATH_INFO: `/v3/projects/9cdb248063644b56b03751405262503b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.407198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cdb248063644b56b03751405262503b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:00.409260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:00.463658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cdb248063644b56b03751405262503b', 'request_id': 'req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:00.471670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cdb248063644b56b03751405262503b', 'request_id': 'req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:00.483548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cdb248063644b56b03751405262503b', 'request_id': 'req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:00.498314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] The token cache is being invalidate because project 9cdb248063644b56b03751405262503b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:00.498703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:00.499267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-760a11e3-ba1c-4d5b-8fbc-ba02b58a27d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:00.501024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1029/2022] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:00 2022] DELETE /identity/v3/projects/9cdb248063644b56b03751405262503b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:00.505808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de1f94-142f-438d-8470-636bcb9dd733 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.506674 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de1f94-142f-438d-8470-636bcb9dd733 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.507153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de1f94-142f-438d-8470-636bcb9dd733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.529259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-60de1f94-142f-438d-8470-636bcb9dd733 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:00.530480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-60de1f94-142f-438d-8470-636bcb9dd733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:00.596282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 994/2023] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:00.605030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f952b2-cb4c-4ec9-bd75-6f1f42c332d1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.605315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f952b2-cb4c-4ec9-bd75-6f1f42c332d1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.605642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f952b2-cb4c-4ec9-bd75-6f1f42c332d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.626847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-35f952b2-cb4c-4ec9-bd75-6f1f42c332d1 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:00.628067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-35f952b2-cb4c-4ec9-bd75-6f1f42c332d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:00.637458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1030/2024] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:00.641743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de1f94-142f-438d-8470-636bcb9dd733 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:00.652163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de1f94-142f-438d-8470-636bcb9dd733 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:00.657174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:00.657874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.658429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.658429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.659096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:00.659459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:00.661124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:00.675959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 995/2025] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:00.681147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f952b2-cb4c-4ec9-bd75-6f1f42c332d1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:00.691048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f952b2-cb4c-4ec9-bd75-6f1f42c332d1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:00.697786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b56e3a-7dd2-4add-87e0-6d646cec422b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:00.698443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b56e3a-7dd2-4add-87e0-6d646cec422b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.698783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b56e3a-7dd2-4add-87e0-6d646cec422b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.699130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b56e3a-7dd2-4add-87e0-6d646cec422b admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.699643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b56e3a-7dd2-4add-87e0-6d646cec422b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:00.703406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b56e3a-7dd2-4add-87e0-6d646cec422b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:00.743933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1031/2026] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:00 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:00.745231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:00.756600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7ca41-b171-4dcd-98b6-b68bd553f458 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:00.793410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:00.866650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ade37-0cfb-497b-8962-aaad40f924d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:00.870632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ade37-0cfb-497b-8962-aaad40f924d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.870969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ade37-0cfb-497b-8962-aaad40f924d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.871208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ade37-0cfb-497b-8962-aaad40f924d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.871878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ade37-0cfb-497b-8962-aaad40f924d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:00.873994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ade37-0cfb-497b-8962-aaad40f924d5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:00.900154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 996/2027] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:00 2022] POST /identity/v3/users => generated 284 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:00.904253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b56e3a-7dd2-4add-87e0-6d646cec422b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:00.920130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b56e3a-7dd2-4add-87e0-6d646cec422b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:00.927033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538689eb-ead8-415e-a79a-b0d1b85359e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:00.927685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538689eb-ead8-415e-a79a-b0d1b85359e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.928564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538689eb-ead8-415e-a79a-b0d1b85359e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.928884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538689eb-ead8-415e-a79a-b0d1b85359e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.929651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538689eb-ead8-415e-a79a-b0d1b85359e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:00.931225 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538689eb-ead8-415e-a79a-b0d1b85359e9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:00.932715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:00.948372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1032/2028] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:00.949054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ade37-0cfb-497b-8962-aaad40f924d5 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:00.965155 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ade37-0cfb-497b-8962-aaad40f924d5 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:00.977335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:00.978050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:00.978435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:00.978784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:00.989623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:00.989623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:00.996272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 997/2029] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:01.000560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538689eb-ead8-415e-a79a-b0d1b85359e9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:01.014818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538689eb-ead8-415e-a79a-b0d1b85359e9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:01.021181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ce7a2-62bf-4e38-a6ab-11f5f30642a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:01.024318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ce7a2-62bf-4e38-a6ab-11f5f30642a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:01.024640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ce7a2-62bf-4e38-a6ab-11f5f30642a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:01.024920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ce7a2-62bf-4e38-a6ab-11f5f30642a2 admin admin] PATH_INFO: `/v3/projects/d57e79a85032404097c1fb618b22b2cf/users/2ae084ecbe32411fa01c77781ded5723/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:01.036004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:01.039431 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:01.052732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc206b0-364e-49fa-88a1-e95b4da68b37 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:01.058564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:01.059466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:01.059997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:01.060114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ce7a2-62bf-4e38-a6ab-11f5f30642a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57e79a85032404097c1fb618b22b2cf, user_id=2ae084ecbe32411fa01c77781ded5723, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:01.060549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] PATH_INFO: `/v3/users/0d9fa00f83b74a2b85d5e6de22af4072` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:01.061908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9fa00f83b74a2b85d5e6de22af4072)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:01.062897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ce7a2-62bf-4e38-a6ab-11f5f30642a2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:01.065087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:01.080549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1033/2030] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:00 2022] PUT /identity/v3/projects/d57e79a85032404097c1fb618b22b2cf/users/2ae084ecbe32411fa01c77781ded5723/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:01.091498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44833b9-9824-4312-b19f-966b1fc86caa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:01.091775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44833b9-9824-4312-b19f-966b1fc86caa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:01.092047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44833b9-9824-4312-b19f-966b1fc86caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:01.123040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9fa00f83b74a2b85d5e6de22af4072'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.125272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b44833b9-9824-4312-b19f-966b1fc86caa None None] MFA Rules not processed for user `2ae084ecbe32411fa01c77781ded5723`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:01.126514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b44833b9-9824-4312-b19f-966b1fc86caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:01.132880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9fa00f83b74a2b85d5e6de22af4072'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.140854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9fa00f83b74a2b85d5e6de22af4072'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.141762 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9fa00f83b74a2b85d5e6de22af4072'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.142328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-81235110-e8de-4ede-9f61-316e53067e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9fa00f83b74a2b85d5e6de22af4072'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.178949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 998/2031] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:01 2022] DELETE /identity/v3/users/0d9fa00f83b74a2b85d5e6de22af4072 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:01.185501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df999b-3e88-4037-9e28-e721cd4c34d5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:01.185756 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df999b-3e88-4037-9e28-e721cd4c34d5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:01.186036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df999b-3e88-4037-9e28-e721cd4c34d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:01.216234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-90df999b-3e88-4037-9e28-e721cd4c34d5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:01.217900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-90df999b-3e88-4037-9e28-e721cd4c34d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:01.228715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1034/2032] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:01.240582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44833b9-9824-4312-b19f-966b1fc86caa None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:01.242922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b44833b9-9824-4312-b19f-966b1fc86caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:01.315372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 999/2033] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:01.324282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df999b-3e88-4037-9e28-e721cd4c34d5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:01.325856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-90df999b-3e88-4037-9e28-e721cd4c34d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:01.333611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44833b9-9824-4312-b19f-966b1fc86caa None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:01.348597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:01.403562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f6e77-21eb-4a1d-ae11-b05dadf5629e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a88817f3a4402290e9358826a63904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:01.403948 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f6e77-21eb-4a1d-ae11-b05dadf5629e service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:01.404120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f6e77-21eb-4a1d-ae11-b05dadf5629e service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:01.404400 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f6e77-21eb-4a1d-ae11-b05dadf5629e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:01.407221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df999b-3e88-4037-9e28-e721cd4c34d5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:01.411127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:01.411739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:01.412019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:01.412924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:01.422346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:01.423793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:01.434106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1000/2034] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:01.436747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:01.466146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f6e77-21eb-4a1d-ae11-b05dadf5629e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:01.468146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f6e77-21eb-4a1d-ae11-b05dadf5629e service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:01.480588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1035/2035] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:01.730324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:01.731847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:01.747420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d9425-9dbc-4b37-8552-b3993303b05a service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:01.753202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:01.753853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:01.754134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:01.754419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] PATH_INFO: `/v3/projects/7c2eaebf216d4ad5b7bf817ba9bbdef6` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:01.755568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c2eaebf216d4ad5b7bf817ba9bbdef6)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:01.758248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:01.817869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c2eaebf216d4ad5b7bf817ba9bbdef6', 'request_id': 'req-c67d0e0e-46f4-48d0-ac87-1356087928e6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.828779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c2eaebf216d4ad5b7bf817ba9bbdef6', 'request_id': 'req-c67d0e0e-46f4-48d0-ac87-1356087928e6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.837997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c2eaebf216d4ad5b7bf817ba9bbdef6', 'request_id': 'req-c67d0e0e-46f4-48d0-ac87-1356087928e6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.850899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] The token cache is being invalidate because project 7c2eaebf216d4ad5b7bf817ba9bbdef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:01.851261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.852172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:01.853814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1001/2036] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:01 2022] DELETE /identity/v3/projects/7c2eaebf216d4ad5b7bf817ba9bbdef6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:01.858017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f6e77-21eb-4a1d-ae11-b05dadf5629e service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:01.869558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f6e77-21eb-4a1d-ae11-b05dadf5629e service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:01.942606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:01.944760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:01.945422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:01.946169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] PATH_INFO: `/v3/users/1beb76e3d7d44af3941176d840714aa6` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:01.947518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=1beb76e3d7d44af3941176d840714aa6)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:01.950825 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:02.043938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1beb76e3d7d44af3941176d840714aa6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.052132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1beb76e3d7d44af3941176d840714aa6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.061809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1beb76e3d7d44af3941176d840714aa6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.062705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1beb76e3d7d44af3941176d840714aa6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.064382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1beb76e3d7d44af3941176d840714aa6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.088033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1036/2037] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:01 2022] DELETE /identity/v3/users/1beb76e3d7d44af3941176d840714aa6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:02.470079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:02.472058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:02.511268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d0e0e-46f4-48d0-ac87-1356087928e6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:02.628004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:02.628822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:02.629101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:02.629372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] PATH_INFO: `/v3/projects/ca313d2530f042459443481d3e9b2c56` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:02.630534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca313d2530f042459443481d3e9b2c56)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:02.632582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:02.681587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:02.688646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca313d2530f042459443481d3e9b2c56', 'request_id': 'req-830fd35c-5a10-4518-98b9-748e6471f661'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.694912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca313d2530f042459443481d3e9b2c56', 'request_id': 'req-830fd35c-5a10-4518-98b9-748e6471f661'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.706073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca313d2530f042459443481d3e9b2c56', 'request_id': 'req-830fd35c-5a10-4518-98b9-748e6471f661'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.718436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] The token cache is being invalidate because project ca313d2530f042459443481d3e9b2c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:02.718730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.719368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.720887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1002/2038] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:02 2022] DELETE /identity/v3/projects/ca313d2530f042459443481d3e9b2c56 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:02.725966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:02.727630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:02.741995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b10c1-b4af-4736-b373-1c1c0f29d5da admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:02.792088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:02.804357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:02.804988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:02.805267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:02.805566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] PATH_INFO: `/v3/users/695c136f22264cbeb6b48a80be400e1a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:02.807247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=695c136f22264cbeb6b48a80be400e1a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:02.810192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:02.859995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695c136f22264cbeb6b48a80be400e1a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.869100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695c136f22264cbeb6b48a80be400e1a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.877237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695c136f22264cbeb6b48a80be400e1a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.877931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695c136f22264cbeb6b48a80be400e1a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.878578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-455e82ca-7f31-4c02-83ff-4995d53c6e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695c136f22264cbeb6b48a80be400e1a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:02.901849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1037/2039] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:02 2022] DELETE /identity/v3/users/695c136f22264cbeb6b48a80be400e1a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:03.209908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:03.211661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:03.224359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd35c-5a10-4518-98b9-748e6471f661 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:03.291335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:03.291986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.292262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.292569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] PATH_INFO: `/v3/projects/97c45cdec2174b5da79430009b67cc84` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.293706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c45cdec2174b5da79430009b67cc84)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:03.295641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:03.347984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c45cdec2174b5da79430009b67cc84', 'request_id': 'req-0a948e18-7a71-4f66-af03-0394ffeed299'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:03.354091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c45cdec2174b5da79430009b67cc84', 'request_id': 'req-0a948e18-7a71-4f66-af03-0394ffeed299'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:03.362608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c45cdec2174b5da79430009b67cc84', 'request_id': 'req-0a948e18-7a71-4f66-af03-0394ffeed299'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:03.374159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] The token cache is being invalidate because project 97c45cdec2174b5da79430009b67cc84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:03.374443 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:03.375080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0a948e18-7a71-4f66-af03-0394ffeed299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:03.376603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1003/2040] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:03 2022] DELETE /identity/v3/projects/97c45cdec2174b5da79430009b67cc84 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:03.382445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1422d-de72-48bc-af78-b52350651910 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.383096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1422d-de72-48bc-af78-b52350651910 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.383096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1422d-de72-48bc-af78-b52350651910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.404960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b4d1422d-de72-48bc-af78-b52350651910 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:03.406174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b4d1422d-de72-48bc-af78-b52350651910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:03.474036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1038/2041] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:03.482351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc22d6e-74e7-4e03-aa38-70f4cb6081a5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.482898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc22d6e-74e7-4e03-aa38-70f4cb6081a5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.482898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc22d6e-74e7-4e03-aa38-70f4cb6081a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.503640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2cc22d6e-74e7-4e03-aa38-70f4cb6081a5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:03.505111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2cc22d6e-74e7-4e03-aa38-70f4cb6081a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:03.513893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1004/2042] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:03.518113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1422d-de72-48bc-af78-b52350651910 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:03.528206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1422d-de72-48bc-af78-b52350651910 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:03.532852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:03.533454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.533738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.534024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.534683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:03.534976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:03.536612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:03.549089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1039/2043] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:03.554127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc22d6e-74e7-4e03-aa38-70f4cb6081a5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:03.565973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc22d6e-74e7-4e03-aa38-70f4cb6081a5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:03.571298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a4d5-1da2-4bff-9c28-078c192015b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:03.571941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a4d5-1da2-4bff-9c28-078c192015b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.572219 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a4d5-1da2-4bff-9c28-078c192015b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.572518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a4d5-1da2-4bff-9c28-078c192015b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.573170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4a4d5-1da2-4bff-9c28-078c192015b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:03.575050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4a4d5-1da2-4bff-9c28-078c192015b3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:03.605967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1005/2044] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:03 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:03.610079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:03.621220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f481a-3a0c-40fa-8b27-4344403686b3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:03.680144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:03.687229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e762a9-58d1-4828-8baa-4826dcb617e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:03.688068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e762a9-58d1-4828-8baa-4826dcb617e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.688600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e762a9-58d1-4828-8baa-4826dcb617e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.688946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e762a9-58d1-4828-8baa-4826dcb617e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.690221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e762a9-58d1-4828-8baa-4826dcb617e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:03.692214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e762a9-58d1-4828-8baa-4826dcb617e6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:03.715764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1040/2045] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:03 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:03.720752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a4d5-1da2-4bff-9c28-078c192015b3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:03.731832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a4d5-1da2-4bff-9c28-078c192015b3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:03.736935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f56eb-9f09-4cfe-a947-77d1c8d2b3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:03.737596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f56eb-9f09-4cfe-a947-77d1c8d2b3de admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.737876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f56eb-9f09-4cfe-a947-77d1c8d2b3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.738166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f56eb-9f09-4cfe-a947-77d1c8d2b3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.738814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f56eb-9f09-4cfe-a947-77d1c8d2b3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:03.740753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f56eb-9f09-4cfe-a947-77d1c8d2b3de admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:03.756628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1006/2046] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:03.757600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e762a9-58d1-4828-8baa-4826dcb617e6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:03.768042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e762a9-58d1-4828-8baa-4826dcb617e6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:03.774176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5803348-1664-4c5a-94e8-27d068351d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:03.775348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5803348-1664-4c5a-94e8-27d068351d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.775776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5803348-1664-4c5a-94e8-27d068351d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.776267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5803348-1664-4c5a-94e8-27d068351d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.777171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5803348-1664-4c5a-94e8-27d068351d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:03.779475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5803348-1664-4c5a-94e8-27d068351d09 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:03.796780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1041/2047] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:03.800649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f56eb-9f09-4cfe-a947-77d1c8d2b3de admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:03.814053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f56eb-9f09-4cfe-a947-77d1c8d2b3de admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:03.827658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:03.828505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.828794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.829237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] PATH_INFO: `/v3/projects/d11b2b1481db4b878ceee550360da8fe/users/eef1ea8d331444e892d0b30c1feef904/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.845136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11b2b1481db4b878ceee550360da8fe, user_id=eef1ea8d331444e892d0b30c1feef904, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:03.847829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:03.851411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:03.863761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1007/2048] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:03 2022] PUT /identity/v3/projects/d11b2b1481db4b878ceee550360da8fe/users/eef1ea8d331444e892d0b30c1feef904/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:03.869587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10772665-8001-4f30-9fa6-7a84643ed7c6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:03.869874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10772665-8001-4f30-9fa6-7a84643ed7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:03.870145 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10772665-8001-4f30-9fa6-7a84643ed7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:03.890081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-10772665-8001-4f30-9fa6-7a84643ed7c6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:03.891292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-10772665-8001-4f30-9fa6-7a84643ed7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:03.954660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1042/2049] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:03.965095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:03.966820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:04.025963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e0c94-7cb6-495f-80b6-de66c54cb1e8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:04.029594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:04.030188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:04.030461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:04.030729 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:04.042029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:04.043461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:04.053690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1008/2050] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:04.891521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fae7a-3691-4924-907d-2d2e7831f6a0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:04.892064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fae7a-3691-4924-907d-2d2e7831f6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:04.892269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fae7a-3691-4924-907d-2d2e7831f6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:04.937361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3c6fae7a-3691-4924-907d-2d2e7831f6a0 None None] MFA Rules not processed for user `eef1ea8d331444e892d0b30c1feef904`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:04.938650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3c6fae7a-3691-4924-907d-2d2e7831f6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:05.061005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1043/2051] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:05.072432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:05.074009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:05.144676 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69b382-0940-4694-b6f8-154af8db14ed service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:05.149308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3952a7-03a7-49de-966c-ab838c268f53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:05.152918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3952a7-03a7-49de-966c-ab838c268f53 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:05.153199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3952a7-03a7-49de-966c-ab838c268f53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:05.153470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3952a7-03a7-49de-966c-ab838c268f53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:05.164170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3952a7-03a7-49de-966c-ab838c268f53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:05.166540 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3952a7-03a7-49de-966c-ab838c268f53 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:05.178678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1009/2052] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:06.742512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fae7a-3691-4924-907d-2d2e7831f6a0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:06.744131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3c6fae7a-3691-4924-907d-2d2e7831f6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:06.758364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fae7a-3691-4924-907d-2d2e7831f6a0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:06.762769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:06.763388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:06.763663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:06.763932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] PATH_INFO: `/v3/users/5794e583b0234d098abec304c567533e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:06.765048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5794e583b0234d098abec304c567533e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:06.766950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:06.814459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5794e583b0234d098abec304c567533e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:06.823073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5794e583b0234d098abec304c567533e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:06.827090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:06.831025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5794e583b0234d098abec304c567533e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:06.831732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5794e583b0234d098abec304c567533e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:06.832421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5794e583b0234d098abec304c567533e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:06.852404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1044/2053] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:06 2022] DELETE /identity/v3/users/5794e583b0234d098abec304c567533e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:06.857052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e2af1-1552-4c1d-afcd-b34c04de0436 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:06.857503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e2af1-1552-4c1d-afcd-b34c04de0436 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:06.857591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e2af1-1552-4c1d-afcd-b34c04de0436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:06.877551 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f17e2af1-1552-4c1d-afcd-b34c04de0436 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:06.879051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f17e2af1-1552-4c1d-afcd-b34c04de0436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:06.942462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1010/2054] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:06.952860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:06.954250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:07.015357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faeb9d-fb96-4d5f-8e70-4ec664f58cb9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:07.019104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4c376-f6eb-4326-b7a6-5e63ab1ef95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:07.019777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4c376-f6eb-4326-b7a6-5e63ab1ef95c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:07.020062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4c376-f6eb-4326-b7a6-5e63ab1ef95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:07.020412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4c376-f6eb-4326-b7a6-5e63ab1ef95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:07.031530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4c376-f6eb-4326-b7a6-5e63ab1ef95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:07.033084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4c376-f6eb-4326-b7a6-5e63ab1ef95c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:07.045369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1045/2055] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:07.433669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e2af1-1552-4c1d-afcd-b34c04de0436 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:07.435359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f17e2af1-1552-4c1d-afcd-b34c04de0436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:07.449409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e2af1-1552-4c1d-afcd-b34c04de0436 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:07.455668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:07.456210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:07.457007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:07.457007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] PATH_INFO: `/v3/projects/095eaf4a0919496daf09665b402a7c09` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:07.461673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=095eaf4a0919496daf09665b402a7c09)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:07.463683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:07.514108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:07.525479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095eaf4a0919496daf09665b402a7c09', 'request_id': 'req-25d391e8-40af-4f0b-b193-c4a42ac73084'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.530872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095eaf4a0919496daf09665b402a7c09', 'request_id': 'req-25d391e8-40af-4f0b-b193-c4a42ac73084'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.538818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095eaf4a0919496daf09665b402a7c09', 'request_id': 'req-25d391e8-40af-4f0b-b193-c4a42ac73084'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.551869 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] The token cache is being invalidate because project 095eaf4a0919496daf09665b402a7c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:07.552299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.553455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25d391e8-40af-4f0b-b193-c4a42ac73084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.555495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1011/2056] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:07 2022] DELETE /identity/v3/projects/095eaf4a0919496daf09665b402a7c09 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:07.559386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4c376-f6eb-4326-b7a6-5e63ab1ef95c service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:07.571946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4c376-f6eb-4326-b7a6-5e63ab1ef95c service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:07.606148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e896c1-917e-49cd-8817-c3a89531853c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:07.606469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e896c1-917e-49cd-8817-c3a89531853c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:07.606750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e896c1-917e-49cd-8817-c3a89531853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:07.632411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-01e896c1-917e-49cd-8817-c3a89531853c None None] MFA Rules not processed for user `eef1ea8d331444e892d0b30c1feef904`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:07.633742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-01e896c1-917e-49cd-8817-c3a89531853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:07.641214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:07.641458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:07.641746 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:07.643235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] PATH_INFO: `/v3/users/abfa6d436339409c98da3bdf4b5767c7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:07.645593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfa6d436339409c98da3bdf4b5767c7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:07.649374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:07.717411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfa6d436339409c98da3bdf4b5767c7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.718903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1012/2057] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:07.728340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfa6d436339409c98da3bdf4b5767c7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.737327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfa6d436339409c98da3bdf4b5767c7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.738742 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfa6d436339409c98da3bdf4b5767c7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.739475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-614cc1c9-fc14-4d0a-9526-49e542715874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfa6d436339409c98da3bdf4b5767c7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:07.759815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:07.767860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1046/2058] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:07 2022] DELETE /identity/v3/users/abfa6d436339409c98da3bdf4b5767c7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:08.309195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e896c1-917e-49cd-8817-c3a89531853c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:08.310885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-01e896c1-917e-49cd-8817-c3a89531853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:08.352112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e896c1-917e-49cd-8817-c3a89531853c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:08.432798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:08.434009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:08.434009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:08.434009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] PATH_INFO: `/v3/projects/4c1bec9f5c03472d94fbe7ba748a3a98` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:08.435128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1bec9f5c03472d94fbe7ba748a3a98)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:08.437802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:08.512772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1bec9f5c03472d94fbe7ba748a3a98', 'request_id': 'req-d0753a41-896b-4943-bea3-13f22e170588'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:08.528577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1bec9f5c03472d94fbe7ba748a3a98', 'request_id': 'req-d0753a41-896b-4943-bea3-13f22e170588'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:08.562055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1bec9f5c03472d94fbe7ba748a3a98', 'request_id': 'req-d0753a41-896b-4943-bea3-13f22e170588'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:08.574338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:08.583461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] The token cache is being invalidate because project 4c1bec9f5c03472d94fbe7ba748a3a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:08.584059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:08.586498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d0753a41-896b-4943-bea3-13f22e170588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:08.592278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1013/2059] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:08 2022] DELETE /identity/v3/projects/4c1bec9f5c03472d94fbe7ba748a3a98 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:08.593981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61ca95-ff8d-451f-b691-b9296b40351e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:08.594513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61ca95-ff8d-451f-b691-b9296b40351e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:08.594898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61ca95-ff8d-451f-b691-b9296b40351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:08.631272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7e61ca95-ff8d-451f-b691-b9296b40351e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:08.636206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7e61ca95-ff8d-451f-b691-b9296b40351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:08.733363 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1047/2060] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:08.747350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c51f3-b0e2-489c-a259-637f2fb4cbe4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:08.747676 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c51f3-b0e2-489c-a259-637f2fb4cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:08.747963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c51f3-b0e2-489c-a259-637f2fb4cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:08.780929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9c5c51f3-b0e2-489c-a259-637f2fb4cbe4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:08.782249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9c5c51f3-b0e2-489c-a259-637f2fb4cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:08.798904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1014/2061] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:08.803989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61ca95-ff8d-451f-b691-b9296b40351e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:08.828014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61ca95-ff8d-451f-b691-b9296b40351e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:08.828014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:08.828014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:08.828014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:08.828380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:08.829225 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:08.829548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:08.831248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:08.848479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1048/2062] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:08.850808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c51f3-b0e2-489c-a259-637f2fb4cbe4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:08.866078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c51f3-b0e2-489c-a259-637f2fb4cbe4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:08.872253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ad619-7b4b-45f2-b072-f04b15351b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:08.873198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad619-7b4b-45f2-b072-f04b15351b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:08.873554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad619-7b4b-45f2-b072-f04b15351b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:08.874953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad619-7b4b-45f2-b072-f04b15351b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:08.874953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ad619-7b4b-45f2-b072-f04b15351b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:08.876766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ad619-7b4b-45f2-b072-f04b15351b6f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:08.921544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1015/2063] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:08 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:08.922507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:08.936993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753590a-625e-4d32-83d3-84a4e13b5f5e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.017734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a966-b712-4e8e-9957-7ff750b45c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.018362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a966-b712-4e8e-9957-7ff750b45c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.018713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a966-b712-4e8e-9957-7ff750b45c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.018921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086a966-b712-4e8e-9957-7ff750b45c47 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.019723 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086a966-b712-4e8e-9957-7ff750b45c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.021945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086a966-b712-4e8e-9957-7ff750b45c47 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.045160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1049/2064] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:08 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:09.050175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ad619-7b4b-45f2-b072-f04b15351b6f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.062518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ad619-7b4b-45f2-b072-f04b15351b6f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.067802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5077b-8094-4fc6-a99e-bb73fd2143e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.069166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5077b-8094-4fc6-a99e-bb73fd2143e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.069751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5077b-8094-4fc6-a99e-bb73fd2143e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.070134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5077b-8094-4fc6-a99e-bb73fd2143e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.070874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5077b-8094-4fc6-a99e-bb73fd2143e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.072608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5077b-8094-4fc6-a99e-bb73fd2143e3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.084017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1016/2065] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:09.088282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a966-b712-4e8e-9957-7ff750b45c47 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.100122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086a966-b712-4e8e-9957-7ff750b45c47 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.105930 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fbab4-be05-4024-b985-0106039f8f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.106536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fbab4-be05-4024-b985-0106039f8f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.106814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fbab4-be05-4024-b985-0106039f8f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.107153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fbab4-be05-4024-b985-0106039f8f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.107803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fbab4-be05-4024-b985-0106039f8f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.109430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fbab4-be05-4024-b985-0106039f8f8c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.122330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1050/2066] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:09.126463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5077b-8094-4fc6-a99e-bb73fd2143e3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.138938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5077b-8094-4fc6-a99e-bb73fd2143e3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.143891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.144937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.145398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.145883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] PATH_INFO: `/v3/projects/71246bfb1d094c22845d62ae0ddb7fc6/users/15918950b65c4d10a21b8174800adc0c/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.169336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71246bfb1d094c22845d62ae0ddb7fc6, user_id=15918950b65c4d10a21b8174800adc0c, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.171979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.190197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1017/2067] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:09 2022] PUT /identity/v3/projects/71246bfb1d094c22845d62ae0ddb7fc6/users/15918950b65c4d10a21b8174800adc0c/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:09.203670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433e6e0-896c-4357-a4a4-23aa9237db95 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.204084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433e6e0-896c-4357-a4a4-23aa9237db95 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.204433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433e6e0-896c-4357-a4a4-23aa9237db95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.221102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.224402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:09.240015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e433e6e0-896c-4357-a4a4-23aa9237db95 None None] MFA Rules not processed for user `15918950b65c4d10a21b8174800adc0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:09.242240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e433e6e0-896c-4357-a4a4-23aa9237db95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:09.313639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd431771-041b-4272-aaf4-dc28165444a3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.324909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.324909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.324909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.324909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] PATH_INFO: `/v3/users/c68d9c44293845b8910cabd9768db429` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.326343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68d9c44293845b8910cabd9768db429)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.328374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.345201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:09.364591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1051/2068] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:09.370176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433e6e0-896c-4357-a4a4-23aa9237db95 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.393687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433e6e0-896c-4357-a4a4-23aa9237db95 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.402819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68d9c44293845b8910cabd9768db429'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:09.417712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.418684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.419932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.420401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.428375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.429504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68d9c44293845b8910cabd9768db429'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:09.432640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.446187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68d9c44293845b8910cabd9768db429'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:09.448139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68d9c44293845b8910cabd9768db429'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:09.448895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-dc3fb024-9f9c-4c8c-9e3b-b505f9802483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68d9c44293845b8910cabd9768db429'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:09.477439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1018/2069] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:09 2022] DELETE /identity/v3/users/c68d9c44293845b8910cabd9768db429 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:09.482037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463bf6-540e-4880-850c-6836e37f7603 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.482322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463bf6-540e-4880-850c-6836e37f7603 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.482750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55463bf6-540e-4880-850c-6836e37f7603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.494083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1052/2070] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:09 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:09.498281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.500021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:09.507520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-55463bf6-540e-4880-850c-6836e37f7603 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:09.509367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-55463bf6-540e-4880-850c-6836e37f7603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:09.520066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:09.580386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67fd7c-6a8d-4201-b2db-b0fddf6715cf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.585265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8499-028b-4d35-92e9-d1b017074808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.585872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8499-028b-4d35-92e9-d1b017074808 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.586261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8499-028b-4d35-92e9-d1b017074808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.586539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8499-028b-4d35-92e9-d1b017074808 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.587188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8499-028b-4d35-92e9-d1b017074808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.588231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1019/2071] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:09.592058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8499-028b-4d35-92e9-d1b017074808 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.598194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55463bf6-540e-4880-850c-6836e37f7603 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.600417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-55463bf6-540e-4880-850c-6836e37f7603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:09.620671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1053/2072] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:09 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:09.622598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8499-028b-4d35-92e9-d1b017074808 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.633893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8499-028b-4d35-92e9-d1b017074808 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.639918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abbe9f-3147-4143-bff1-297163a783e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.640655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abbe9f-3147-4143-bff1-297163a783e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.640936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abbe9f-3147-4143-bff1-297163a783e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.641207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abbe9f-3147-4143-bff1-297163a783e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.641859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abbe9f-3147-4143-bff1-297163a783e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.643474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abbe9f-3147-4143-bff1-297163a783e4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.656509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1054/2073] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:09.659694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abbe9f-3147-4143-bff1-297163a783e4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.674577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abbe9f-3147-4143-bff1-297163a783e4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.683235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee08ae5-6d45-41f3-bf3e-d1add3ad24e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.684019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55463bf6-540e-4880-850c-6836e37f7603 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.684145 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee08ae5-6d45-41f3-bf3e-d1add3ad24e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.684464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee08ae5-6d45-41f3-bf3e-d1add3ad24e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.684880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee08ae5-6d45-41f3-bf3e-d1add3ad24e6 admin admin] PATH_INFO: `/v3/projects/18cd433662cc446ea193873f698ad8dc/users/d82db997b0dc4d08aef6666f1cf6cf84/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.696071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0cb41-8e13-44e7-b918-fa4505cd87a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.696675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cb41-8e13-44e7-b918-fa4505cd87a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.696962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cb41-8e13-44e7-b918-fa4505cd87a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.697240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cb41-8e13-44e7-b918-fa4505cd87a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.711461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0cb41-8e13-44e7-b918-fa4505cd87a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.712060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee08ae5-6d45-41f3-bf3e-d1add3ad24e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cd433662cc446ea193873f698ad8dc, user_id=d82db997b0dc4d08aef6666f1cf6cf84, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.713624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0cb41-8e13-44e7-b918-fa4505cd87a1 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.715102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee08ae5-6d45-41f3-bf3e-d1add3ad24e6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.730399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1020/2074] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:09.741017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1055/2075] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:09 2022] PUT /identity/v3/projects/18cd433662cc446ea193873f698ad8dc/users/d82db997b0dc4d08aef6666f1cf6cf84/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:09.745046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0cb41-8e13-44e7-b918-fa4505cd87a1 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.756941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0cb41-8e13-44e7-b918-fa4505cd87a1 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.868413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bda03-2431-4b9f-a4ca-5825d18e61c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.869299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bda03-2431-4b9f-a4ca-5825d18e61c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.869733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bda03-2431-4b9f-a4ca-5825d18e61c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.870110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bda03-2431-4b9f-a4ca-5825d18e61c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.870914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bda03-2431-4b9f-a4ca-5825d18e61c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.872648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bda03-2431-4b9f-a4ca-5825d18e61c4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.885699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1021/2076] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:09.890759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee08ae5-6d45-41f3-bf3e-d1add3ad24e6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:09.903643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee08ae5-6d45-41f3-bf3e-d1add3ad24e6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:09.908534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:09.909461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.909803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.910146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] PATH_INFO: `/v3/projects/18cd433662cc446ea193873f698ad8dc/users/d82db997b0dc4d08aef6666f1cf6cf84/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.912181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cd433662cc446ea193873f698ad8dc, user_id=d82db997b0dc4d08aef6666f1cf6cf84, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:09.915166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:09.930569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1056/2077] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:09 2022] PUT /identity/v3/projects/18cd433662cc446ea193873f698ad8dc/users/d82db997b0dc4d08aef6666f1cf6cf84/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:09.942461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d581876-a881-472e-bcb4-e1d793861cb2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:09.942461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d581876-a881-472e-bcb4-e1d793861cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:09.942675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d581876-a881-472e-bcb4-e1d793861cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:09.990024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1d581876-a881-472e-bcb4-e1d793861cb2 None None] MFA Rules not processed for user `d82db997b0dc4d08aef6666f1cf6cf84`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:09.991907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1d581876-a881-472e-bcb4-e1d793861cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.057704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:10.092666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1022/2078] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:10.111411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.113384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.189023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f0908-d27f-44fe-be50-f2e198d00090 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.192802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.194285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.194742 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.195291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.199788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d581876-a881-472e-bcb4-e1d793861cb2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.201447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1d581876-a881-472e-bcb4-e1d793861cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.207138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.208641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.220044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1057/2079] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:10.220911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.222730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.259448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:10.309287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d581876-a881-472e-bcb4-e1d793861cb2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.317109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.317672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.317905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.318177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] PATH_INFO: `/v3/users/2d11f2ede1204e81b02567334881801f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.319372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d11f2ede1204e81b02567334881801f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.321432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.327414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cb6cc-d35f-4a9e-a91c-0af0b824aef7 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.333041 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.334024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.334452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.334869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] PATH_INFO: `/v3/projects/c355ef41e7974995b022c3f5a6e23295` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.336187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c355ef41e7974995b022c3f5a6e23295)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.338411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.380474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d11f2ede1204e81b02567334881801f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.390343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d11f2ede1204e81b02567334881801f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.399215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d11f2ede1204e81b02567334881801f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.400139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d11f2ede1204e81b02567334881801f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.400914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-58f3fa04-039b-4d26-b432-fee5fae428f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d11f2ede1204e81b02567334881801f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.405796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c355ef41e7974995b022c3f5a6e23295', 'request_id': 'req-7e9408a0-4601-4824-aef3-134f9ea52510'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.413435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c355ef41e7974995b022c3f5a6e23295', 'request_id': 'req-7e9408a0-4601-4824-aef3-134f9ea52510'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.422474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1023/2080] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:10 2022] DELETE /identity/v3/users/2d11f2ede1204e81b02567334881801f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:10.425875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c355ef41e7974995b022c3f5a6e23295', 'request_id': 'req-7e9408a0-4601-4824-aef3-134f9ea52510'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.428419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34b0d-ebea-4585-9ef4-6be9f35e1e83 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.429074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34b0d-ebea-4585-9ef4-6be9f35e1e83 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.429575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34b0d-ebea-4585-9ef4-6be9f35e1e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.441002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] The token cache is being invalidate because project c355ef41e7974995b022c3f5a6e23295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:10.441286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.441954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7e9408a0-4601-4824-aef3-134f9ea52510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:10.443640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1058/2081] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:10 2022] DELETE /identity/v3/projects/c355ef41e7974995b022c3f5a6e23295 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:10.448862 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d0cb-f5b7-4c51-b34e-a22c498f6fc0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.449234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d0cb-f5b7-4c51-b34e-a22c498f6fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.449645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d0cb-f5b7-4c51-b34e-a22c498f6fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.453518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2ec34b0d-ebea-4585-9ef4-6be9f35e1e83 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:10.454803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2ec34b0d-ebea-4585-9ef4-6be9f35e1e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.476009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c4d6d0cb-f5b7-4c51-b34e-a22c498f6fc0 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:10.479166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c4d6d0cb-f5b7-4c51-b34e-a22c498f6fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.540944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:10.553565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1024/2082] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:10.565604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec34b0d-ebea-4585-9ef4-6be9f35e1e83 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.567246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1059/2083] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:10.567412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2ec34b0d-ebea-4585-9ef4-6be9f35e1e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.575191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe4023-4ac1-4171-87e1-86ce4b0616a9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.575479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe4023-4ac1-4171-87e1-86ce4b0616a9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.575741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe4023-4ac1-4171-87e1-86ce4b0616a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.595762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e3fe4023-4ac1-4171-87e1-86ce4b0616a9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:10.597281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e3fe4023-4ac1-4171-87e1-86ce4b0616a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.606842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1060/2084] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:10.610976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe4023-4ac1-4171-87e1-86ce4b0616a9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.621081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe4023-4ac1-4171-87e1-86ce4b0616a9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.628064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.630381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.630773 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.631011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.631764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:10.632071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.633887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.646081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1061/2085] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:15:10.648079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.650012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:10.651556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec34b0d-ebea-4585-9ef4-6be9f35e1e83 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.655185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d6ec-c1b3-427e-b7b4-26ac6e54171f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.655796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d6ec-c1b3-427e-b7b4-26ac6e54171f service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.656074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d6ec-c1b3-427e-b7b4-26ac6e54171f service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.656344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d6ec-c1b3-427e-b7b4-26ac6e54171f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.667014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d6ec-c1b3-427e-b7b4-26ac6e54171f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.668482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d6ec-c1b3-427e-b7b4-26ac6e54171f service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.678579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1025/2086] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:10.679438 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d6ec-c1b3-427e-b7b4-26ac6e54171f service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.685810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:10.689805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d6ec-c1b3-427e-b7b4-26ac6e54171f service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.696014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c4515-de48-4c92-98ab-71cef9802739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.696872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c4515-de48-4c92-98ab-71cef9802739 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.696989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c4515-de48-4c92-98ab-71cef9802739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.697184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c4515-de48-4c92-98ab-71cef9802739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.697803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c4515-de48-4c92-98ab-71cef9802739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.699763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c4515-de48-4c92-98ab-71cef9802739 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.734514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1026/2087] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:10 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:10.735259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f2a70-54c8-4218-bc4f-eca0c121794c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.738950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c4515-de48-4c92-98ab-71cef9802739 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.811803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c4515-de48-4c92-98ab-71cef9802739 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.820663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66356-7891-4d33-b1bd-fd40abc59fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.821285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66356-7891-4d33-b1bd-fd40abc59fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.821596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66356-7891-4d33-b1bd-fd40abc59fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.821884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66356-7891-4d33-b1bd-fd40abc59fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.822514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a66356-7891-4d33-b1bd-fd40abc59fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.824516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a66356-7891-4d33-b1bd-fd40abc59fc0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.825375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82db997b0dc4d08aef6666f1cf6cf84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18cd433662cc446ea193873f698ad8dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.825989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.826264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.826535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.827659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:10.827659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.829222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.845552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1062/2088] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:10.855284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.863941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1027/2089] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:10 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:10.868544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66356-7891-4d33-b1bd-fd40abc59fc0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.869113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e50fb-5fb1-4fea-bb5e-e7cadd8e77b4 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.875789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddd9bb-43e3-4aac-8353-fdfde933b11d tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82db997b0dc4d08aef6666f1cf6cf84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18cd433662cc446ea193873f698ad8dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.876807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd9bb-43e3-4aac-8353-fdfde933b11d tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.876985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd9bb-43e3-4aac-8353-fdfde933b11d tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.877257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddd9bb-43e3-4aac-8353-fdfde933b11d tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.878399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddd9bb-43e3-4aac-8353-fdfde933b11d tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.879896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddd9bb-43e3-4aac-8353-fdfde933b11d tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.883835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66356-7891-4d33-b1bd-fd40abc59fc0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:10.890655 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66d97a-c63a-4069-9fe5-f62e6b04351a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:10.891392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d97a-c63a-4069-9fe5-f62e6b04351a admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:10.892047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d97a-c63a-4069-9fe5-f62e6b04351a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:10.892326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d97a-c63a-4069-9fe5-f62e6b04351a admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:10.893091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66d97a-c63a-4069-9fe5-f62e6b04351a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:10.894707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66d97a-c63a-4069-9fe5-f62e6b04351a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:10.918919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1028/2090] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:10.920899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66d97a-c63a-4069-9fe5-f62e6b04351a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.926365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1063/2091] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:10 2022] POST /identity/v3/projects => generated 355 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:10.935139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddd9bb-43e3-4aac-8353-fdfde933b11d tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:10.949808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66d97a-c63a-4069-9fe5-f62e6b04351a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:11.036890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc176e32-bdf0-4129-9125-373c010ab1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:11.040084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc176e32-bdf0-4129-9125-373c010ab1de admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.040084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc176e32-bdf0-4129-9125-373c010ab1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.040084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc176e32-bdf0-4129-9125-373c010ab1de admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.040084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc176e32-bdf0-4129-9125-373c010ab1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:11.041354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc176e32-bdf0-4129-9125-373c010ab1de admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:11.043296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:11.050291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddd9bb-43e3-4aac-8353-fdfde933b11d tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:11.057017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82db997b0dc4d08aef6666f1cf6cf84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18cd433662cc446ea193873f698ad8dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:11.058226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.058347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1029/2092] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:11.058546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.058962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.060251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:11.060251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:11.062469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:11.063040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc176e32-bdf0-4129-9125-373c010ab1de admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:11.075909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc176e32-bdf0-4129-9125-373c010ab1de admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:11.077238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1064/2093] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:11.084275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd13b-4fb1-4c28-b6f6-3e0c3ceda231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:11.084993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd13b-4fb1-4c28-b6f6-3e0c3ceda231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.085286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd13b-4fb1-4c28-b6f6-3e0c3ceda231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.085555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd13b-4fb1-4c28-b6f6-3e0c3ceda231 admin admin] PATH_INFO: `/v3/projects/37a9557289ea4fe9903f6cf421d937d1/users/871ff162be0b43c98b531e860f6a02e5/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.110186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fd13b-4fb1-4c28-b6f6-3e0c3ceda231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a9557289ea4fe9903f6cf421d937d1, user_id=871ff162be0b43c98b531e860f6a02e5, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:11.113147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fd13b-4fb1-4c28-b6f6-3e0c3ceda231 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:11.124747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:11.141675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:11.156525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1030/2094] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:11 2022] PUT /identity/v3/projects/37a9557289ea4fe9903f6cf421d937d1/users/871ff162be0b43c98b531e860f6a02e5/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:11.156525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd13b-4fb1-4c28-b6f6-3e0c3ceda231 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:11.186182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee59b8b-45e7-44ce-a22b-5300295b39d3 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:11.238197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:11.239913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd13b-4fb1-4c28-b6f6-3e0c3ceda231 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:11.245547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82db997b0dc4d08aef6666f1cf6cf84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18cd433662cc446ea193873f698ad8dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:11.246305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.246734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.247094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] PATH_INFO: `/v3/projects/31101f72967c4a6b81c02e1b95c5f27f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.248359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Authorizing `identity:delete_project(project_id=31101f72967c4a6b81c02e1b95c5f27f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:11.250388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:11.269601 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:11.270402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.270688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.270983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] PATH_INFO: `/v3/projects/bdcb2a05bd374645bc9393ab1061a20e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.272770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdcb2a05bd374645bc9393ab1061a20e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:11.274694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:11.327741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31101f72967c4a6b81c02e1b95c5f27f', 'request_id': 'req-da9b3118-3e5f-42a9-8f84-7890e332ae93'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.337739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31101f72967c4a6b81c02e1b95c5f27f', 'request_id': 'req-da9b3118-3e5f-42a9-8f84-7890e332ae93'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.357077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31101f72967c4a6b81c02e1b95c5f27f', 'request_id': 'req-da9b3118-3e5f-42a9-8f84-7890e332ae93'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.368491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdcb2a05bd374645bc9393ab1061a20e', 'request_id': 'req-e200bfd6-f383-457d-aae3-24c72821a8f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.373488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] The token cache is being invalidate because project 31101f72967c4a6b81c02e1b95c5f27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:11.374173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.374916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-da9b3118-3e5f-42a9-8f84-7890e332ae93 tempest-VolumeQuotaClassesTest-2005117353 tempest-VolumeQuotaClassesTest-2005117353-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.375206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdcb2a05bd374645bc9393ab1061a20e', 'request_id': 'req-e200bfd6-f383-457d-aae3-24c72821a8f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.376486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1031/2095] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:11 2022] DELETE /identity/v3/projects/31101f72967c4a6b81c02e1b95c5f27f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:11.377840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556a8c8-1534-4e99-bb7b-aace1d3f8aa1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.378123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556a8c8-1534-4e99-bb7b-aace1d3f8aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.378396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556a8c8-1534-4e99-bb7b-aace1d3f8aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.417273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e556a8c8-1534-4e99-bb7b-aace1d3f8aa1 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:11.417892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdcb2a05bd374645bc9393ab1061a20e', 'request_id': 'req-e200bfd6-f383-457d-aae3-24c72821a8f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.418824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e556a8c8-1534-4e99-bb7b-aace1d3f8aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:11.439641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] The token cache is being invalidate because project bdcb2a05bd374645bc9393ab1061a20e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:11.439991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.441451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.446406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1065/2096] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:11 2022] DELETE /identity/v3/projects/bdcb2a05bd374645bc9393ab1061a20e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:11.447467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:11.449322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:11.506339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1032/2097] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:11.520298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556a8c8-1534-4e99-bb7b-aace1d3f8aa1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:11.523662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e556a8c8-1534-4e99-bb7b-aace1d3f8aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:11.533252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:11.550898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200bfd6-f383-457d-aae3-24c72821a8f4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:11.558630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:11.559376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.560210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.560412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] PATH_INFO: `/v3/users/f64fd04b4d76461e80efbcd3e41e1a13` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.561652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64fd04b4d76461e80efbcd3e41e1a13)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:11.564640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:11.654506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64fd04b4d76461e80efbcd3e41e1a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.666199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556a8c8-1534-4e99-bb7b-aace1d3f8aa1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:11.667390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64fd04b4d76461e80efbcd3e41e1a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.672394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:11.673171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.673565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.674058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.677508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64fd04b4d76461e80efbcd3e41e1a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.679354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64fd04b4d76461e80efbcd3e41e1a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.680431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f284a66c-4fb6-44ee-a8d3-25936cd19d97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64fd04b4d76461e80efbcd3e41e1a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.687969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:11.690372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:11.695127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:11.704889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1033/2098] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:11.716502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1066/2099] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:11 2022] DELETE /identity/v3/users/f64fd04b4d76461e80efbcd3e41e1a13 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:11.736116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:11.737848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:11.753918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db97c23-d658-4c7e-95f3-8105371389fe service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:11.843855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:11.844619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.844902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.845174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] PATH_INFO: `/v3/users/15918950b65c4d10a21b8174800adc0c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.847140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15918950b65c4d10a21b8174800adc0c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:11.848397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:11.909832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15918950b65c4d10a21b8174800adc0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.920969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15918950b65c4d10a21b8174800adc0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.934877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15918950b65c4d10a21b8174800adc0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.935627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15918950b65c4d10a21b8174800adc0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.936426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15918950b65c4d10a21b8174800adc0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:11.963755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1034/2100] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:11 2022] DELETE /identity/v3/users/15918950b65c4d10a21b8174800adc0c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:11.968207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d51d8-9ecb-4b10-8250-bad0c44c5e97 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:11.968742 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d51d8-9ecb-4b10-8250-bad0c44c5e97 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:11.968823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d51d8-9ecb-4b10-8250-bad0c44c5e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:11.991679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a04d51d8-9ecb-4b10-8250-bad0c44c5e97 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:11.992925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a04d51d8-9ecb-4b10-8250-bad0c44c5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:12.126310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1067/2101] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:12.200113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:12.202316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:12.249017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:12.293065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf191e-9bc3-4c7b-957d-883743ccfa04 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:12.297318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b1673-b14d-4794-a97e-69508b6526a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:12.297938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b1673-b14d-4794-a97e-69508b6526a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:12.298217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b1673-b14d-4794-a97e-69508b6526a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:12.298603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b1673-b14d-4794-a97e-69508b6526a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:12.316147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b1673-b14d-4794-a97e-69508b6526a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:12.318127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b1673-b14d-4794-a97e-69508b6526a8 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:12.329690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1035/2102] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:12.637149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d51d8-9ecb-4b10-8250-bad0c44c5e97 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:12.638607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a04d51d8-9ecb-4b10-8250-bad0c44c5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:12.661327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d51d8-9ecb-4b10-8250-bad0c44c5e97 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:12.669044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:12.669883 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:12.670169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:12.670442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] PATH_INFO: `/v3/projects/e8121ce17e244a7697059afa1ef42a86` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:12.671842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8121ce17e244a7697059afa1ef42a86)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:12.675471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:12.784162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8121ce17e244a7697059afa1ef42a86', 'request_id': 'req-8efe2004-e0cd-404f-a759-aadad528be03'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:12.796289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8121ce17e244a7697059afa1ef42a86', 'request_id': 'req-8efe2004-e0cd-404f-a759-aadad528be03'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:12.816419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8121ce17e244a7697059afa1ef42a86', 'request_id': 'req-8efe2004-e0cd-404f-a759-aadad528be03'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:12.833454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] The token cache is being invalidate because project e8121ce17e244a7697059afa1ef42a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:12.834410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:12.834994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8efe2004-e0cd-404f-a759-aadad528be03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:12.836984 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1068/2103] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:12 2022] DELETE /identity/v3/projects/e8121ce17e244a7697059afa1ef42a86 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:12.841740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c2026-7a2d-4e1a-a6b0-52cd6c78eed8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:12.842300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c2026-7a2d-4e1a-a6b0-52cd6c78eed8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:12.842839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c2026-7a2d-4e1a-a6b0-52cd6c78eed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:12.869338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5d8c2026-7a2d-4e1a-a6b0-52cd6c78eed8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:12.870660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5d8c2026-7a2d-4e1a-a6b0-52cd6c78eed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:12.988731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1036/2104] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:12.998307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531af89-09fc-42a7-b334-7cee7e14af1a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:12.998904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531af89-09fc-42a7-b334-7cee7e14af1a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:12.999092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531af89-09fc-42a7-b334-7cee7e14af1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.033869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0531af89-09fc-42a7-b334-7cee7e14af1a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:13.037267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0531af89-09fc-42a7-b334-7cee7e14af1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:13.050463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1069/2105] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:13.055848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c2026-7a2d-4e1a-a6b0-52cd6c78eed8 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:13.069269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c2026-7a2d-4e1a-a6b0-52cd6c78eed8 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:13.075421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:13.076781 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:13.077174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:13.077514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.078374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:13.078643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:13.080917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:13.100572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1037/2106] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:13.106180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531af89-09fc-42a7-b334-7cee7e14af1a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:13.121828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531af89-09fc-42a7-b334-7cee7e14af1a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:13.129216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61818fe-8d59-4d99-80f2-589abb24f7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:13.130165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61818fe-8d59-4d99-80f2-589abb24f7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:13.130669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61818fe-8d59-4d99-80f2-589abb24f7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:13.131165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61818fe-8d59-4d99-80f2-589abb24f7c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.132141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61818fe-8d59-4d99-80f2-589abb24f7c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:13.134730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61818fe-8d59-4d99-80f2-589abb24f7c7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:13.169056 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:13.171974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:13.205015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1070/2107] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:13 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:13.208135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61818fe-8d59-4d99-80f2-589abb24f7c7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:13.218921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22cd56-9573-418a-86e5-ec56a102afb3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:13.321967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:13.329822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:13.330466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:13.330466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] PATH_INFO: `/v3/projects/71246bfb1d094c22845d62ae0ddb7fc6` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.337606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71246bfb1d094c22845d62ae0ddb7fc6)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:13.342539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:13.401656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:13.457874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61818fe-8d59-4d99-80f2-589abb24f7c7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:13.470901 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33114d-75ed-4b81-ac26-4c960596b836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:13.472874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33114d-75ed-4b81-ac26-4c960596b836 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:13.473335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33114d-75ed-4b81-ac26-4c960596b836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:13.474251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33114d-75ed-4b81-ac26-4c960596b836 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.483864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33114d-75ed-4b81-ac26-4c960596b836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:13.488347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33114d-75ed-4b81-ac26-4c960596b836 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:13.495447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71246bfb1d094c22845d62ae0ddb7fc6', 'request_id': 'req-5ba153a6-45ed-4552-b7c0-59eff628fc49'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.509470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71246bfb1d094c22845d62ae0ddb7fc6', 'request_id': 'req-5ba153a6-45ed-4552-b7c0-59eff628fc49'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.513873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:13.524479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71246bfb1d094c22845d62ae0ddb7fc6', 'request_id': 'req-5ba153a6-45ed-4552-b7c0-59eff628fc49'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.561261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1071/2108] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:13 2022] POST /identity/v3/users => generated 295 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:13.562787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c5c8-d847-4f53-ac85-f828d96236e9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:13.563168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c5c8-d847-4f53-ac85-f828d96236e9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:13.563594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c5c8-d847-4f53-ac85-f828d96236e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.573374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] The token cache is being invalidate because project 71246bfb1d094c22845d62ae0ddb7fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:13.573675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.574604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.581430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1038/2109] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:13 2022] DELETE /identity/v3/projects/71246bfb1d094c22845d62ae0ddb7fc6 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:13.582271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:13.586992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:13.627647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1c57c5c8-d847-4f53-ac85-f828d96236e9 None None] MFA Rules not processed for user `871ff162be0b43c98b531e860f6a02e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:13.628996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1c57c5c8-d847-4f53-ac85-f828d96236e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:13.733667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba153a6-45ed-4552-b7c0-59eff628fc49 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:13.749645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757073b-5b4b-402d-98db-3bb529e94d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:13.750381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757073b-5b4b-402d-98db-3bb529e94d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:13.750733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757073b-5b4b-402d-98db-3bb529e94d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:13.751076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757073b-5b4b-402d-98db-3bb529e94d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.752030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757073b-5b4b-402d-98db-3bb529e94d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:13.761005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757073b-5b4b-402d-98db-3bb529e94d0f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:13.770544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1072/2110] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:13.770544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57c5c8-d847-4f53-ac85-f828d96236e9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:13.773189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1c57c5c8-d847-4f53-ac85-f828d96236e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:13.788804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1039/2111] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:13.793129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757073b-5b4b-402d-98db-3bb529e94d0f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:13.808483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57c5c8-d847-4f53-ac85-f828d96236e9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:13.823270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:13.823926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:13.824225 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:13.824516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] PATH_INFO: `/v3/users/d82db997b0dc4d08aef6666f1cf6cf84` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.827604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d82db997b0dc4d08aef6666f1cf6cf84)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:13.829619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:13.838940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757073b-5b4b-402d-98db-3bb529e94d0f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:13.855230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:13.855961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:13.856318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:13.856683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:13.857417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:13.859189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:13.884814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1040/2112] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:13.884814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:13.887339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:13.922458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd82db997b0dc4d08aef6666f1cf6cf84'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.935608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd82db997b0dc4d08aef6666f1cf6cf84'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.948853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd82db997b0dc4d08aef6666f1cf6cf84'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.950215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd82db997b0dc4d08aef6666f1cf6cf84'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.951066 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd82db997b0dc4d08aef6666f1cf6cf84'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:13.999160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7395df-5c83-4d8d-8316-ec8671fec6e8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:13.999975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1073/2113] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:13 2022] DELETE /identity/v3/users/d82db997b0dc4d08aef6666f1cf6cf84 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:14.001243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:14.003150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:14.085606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:14.106899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.107663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.107944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.108286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.110135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:14.122236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93192d25-d6bc-463e-be53-6f7a27bbff54 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:14.129835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a821af-146e-4ab3-9806-48f61c5cb5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.131619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a821af-146e-4ab3-9806-48f61c5cb5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.132791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a821af-146e-4ab3-9806-48f61c5cb5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.133264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a821af-146e-4ab3-9806-48f61c5cb5ee admin admin] PATH_INFO: `/v3/projects/2f5caeb5fe654297a25764b72a17762b/users/d797a449db2e4a41a5cdf0772062c955/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.160093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:14.161061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a821af-146e-4ab3-9806-48f61c5cb5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5caeb5fe654297a25764b72a17762b, user_id=d797a449db2e4a41a5cdf0772062c955, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.165810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a821af-146e-4ab3-9806-48f61c5cb5ee admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.184582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1074/2114] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:13 2022] PUT /identity/v3/projects/2f5caeb5fe654297a25764b72a17762b/users/d797a449db2e4a41a5cdf0772062c955/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:14.196521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69529f-b09c-4e7a-9cbb-1f304cd2e5b9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.196744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69529f-b09c-4e7a-9cbb-1f304cd2e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.197116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69529f-b09c-4e7a-9cbb-1f304cd2e5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.198062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.199373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.212122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1041/2115] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:14.226175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ae69529f-b09c-4e7a-9cbb-1f304cd2e5b9 None None] MFA Rules not processed for user `d797a449db2e4a41a5cdf0772062c955`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:14.227386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ae69529f-b09c-4e7a-9cbb-1f304cd2e5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:14.331175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1075/2116] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:14.337961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:14.364229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8599b-480f-489e-b895-59e27077abee service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:14.452123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e6efa-3fe0-4c95-a5d7-6965e211f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.453029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e6efa-3fe0-4c95-a5d7-6965e211f898 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.453386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e6efa-3fe0-4c95-a5d7-6965e211f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.453659 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e6efa-3fe0-4c95-a5d7-6965e211f898 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.454359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e6efa-3fe0-4c95-a5d7-6965e211f898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.456720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e6efa-3fe0-4c95-a5d7-6965e211f898 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.501061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1042/2117] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:14 2022] POST /identity/v3/projects => generated 389 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:14.502036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69529f-b09c-4e7a-9cbb-1f304cd2e5b9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:14.513872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69529f-b09c-4e7a-9cbb-1f304cd2e5b9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:14.598683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ce01b-02ba-4635-9c6a-5130436d7b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.599426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ce01b-02ba-4635-9c6a-5130436d7b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.599911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ce01b-02ba-4635-9c6a-5130436d7b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.600181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ce01b-02ba-4635-9c6a-5130436d7b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.601041 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ce01b-02ba-4635-9c6a-5130436d7b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.602961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ce01b-02ba-4635-9c6a-5130436d7b74 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.631507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1076/2118] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:14 2022] POST /identity/v3/users => generated 301 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:14.636385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e6efa-3fe0-4c95-a5d7-6965e211f898 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:14.650955 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e6efa-3fe0-4c95-a5d7-6965e211f898 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:14.670295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414ee9a-c0b5-470d-ba86-cf747170ef84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.671415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414ee9a-c0b5-470d-ba86-cf747170ef84 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.671765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414ee9a-c0b5-470d-ba86-cf747170ef84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.672103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414ee9a-c0b5-470d-ba86-cf747170ef84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.673066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414ee9a-c0b5-470d-ba86-cf747170ef84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.675182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414ee9a-c0b5-470d-ba86-cf747170ef84 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.691339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1043/2119] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:14.693340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ce01b-02ba-4635-9c6a-5130436d7b74 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:14.707958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ce01b-02ba-4635-9c6a-5130436d7b74 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:14.713067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.713774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.714051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.714326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] PATH_INFO: `/v3/projects/6cfadf687dc140bdad8a4dc06a21009b/users/65276929ce6d4c9798ceb896e13844c1/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.731452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfadf687dc140bdad8a4dc06a21009b, user_id=65276929ce6d4c9798ceb896e13844c1, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.734211 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.750514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1077/2120] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:14 2022] PUT /identity/v3/projects/6cfadf687dc140bdad8a4dc06a21009b/users/65276929ce6d4c9798ceb896e13844c1/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:14.754338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414ee9a-c0b5-470d-ba86-cf747170ef84 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:14.769584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414ee9a-c0b5-470d-ba86-cf747170ef84 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:14.844806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9f7ce-5a2d-4e21-92fb-7572a032fa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.845696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9f7ce-5a2d-4e21-92fb-7572a032fa5e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.845978 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9f7ce-5a2d-4e21-92fb-7572a032fa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.846249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9f7ce-5a2d-4e21-92fb-7572a032fa5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.847277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9f7ce-5a2d-4e21-92fb-7572a032fa5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.850023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9f7ce-5a2d-4e21-92fb-7572a032fa5e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.870329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1044/2121] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:14.871848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:14.873882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:14.886189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9f7ce-5a2d-4e21-92fb-7572a032fa5e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:14.909048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f902a5-7a5f-4f56-9d35-3f659d5e90a5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:14.917163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.918108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.918480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.918904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] PATH_INFO: `/v3/projects/18cd433662cc446ea193873f698ad8dc` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.920392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] RBAC: Authorizing `identity:delete_project(project_id=18cd433662cc446ea193873f698ad8dc)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.922783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.926309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9f7ce-5a2d-4e21-92fb-7572a032fa5e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:14.938260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:14.941028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5eabd-d6ae-4ed3-8fe6-a384f0782286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:14.941875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5eabd-d6ae-4ed3-8fe6-a384f0782286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.942323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5eabd-d6ae-4ed3-8fe6-a384f0782286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.942767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5eabd-d6ae-4ed3-8fe6-a384f0782286 admin admin] PATH_INFO: `/v3/projects/6cfadf687dc140bdad8a4dc06a21009b/users/65276929ce6d4c9798ceb896e13844c1/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:14.945552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5eabd-d6ae-4ed3-8fe6-a384f0782286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfadf687dc140bdad8a4dc06a21009b, user_id=65276929ce6d4c9798ceb896e13844c1, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:14.949498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5eabd-d6ae-4ed3-8fe6-a384f0782286 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:14.961656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1045/2122] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:14 2022] PUT /identity/v3/projects/6cfadf687dc140bdad8a4dc06a21009b/users/65276929ce6d4c9798ceb896e13844c1/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:14.977062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446c7ac-85b9-4890-a1ef-17d057cf55ec None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:14.977665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446c7ac-85b9-4890-a1ef-17d057cf55ec None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:14.978420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446c7ac-85b9-4890-a1ef-17d057cf55ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.001528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18cd433662cc446ea193873f698ad8dc', 'request_id': 'req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.010909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18cd433662cc446ea193873f698ad8dc', 'request_id': 'req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.025397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18cd433662cc446ea193873f698ad8dc', 'request_id': 'req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.033108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b446c7ac-85b9-4890-a1ef-17d057cf55ec None None] MFA Rules not processed for user `65276929ce6d4c9798ceb896e13844c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:15.034774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b446c7ac-85b9-4890-a1ef-17d057cf55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.054741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] The token cache is being invalidate because project 18cd433662cc446ea193873f698ad8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:15.055743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.057013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9a4e1525-a426-42ae-a3db-9a2a8cf7c54c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.061705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1078/2123] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:14 2022] DELETE /identity/v3/projects/18cd433662cc446ea193873f698ad8dc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:15.069936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39723e0b-800e-4895-9c32-541520a1612c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.070509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39723e0b-800e-4895-9c32-541520a1612c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.070889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39723e0b-800e-4895-9c32-541520a1612c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.098116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:15.102484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-39723e0b-800e-4895-9c32-541520a1612c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:15.104943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-39723e0b-800e-4895-9c32-541520a1612c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.192430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1079/2124] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:15.214244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd94417-9a4e-495f-bd4a-55441abe6eb5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.214665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd94417-9a4e-495f-bd4a-55441abe6eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.215017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd94417-9a4e-495f-bd4a-55441abe6eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.216945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1046/2125] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:15.230730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446c7ac-85b9-4890-a1ef-17d057cf55ec None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.232954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b446c7ac-85b9-4890-a1ef-17d057cf55ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.243970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cdd94417-9a4e-495f-bd4a-55441abe6eb5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:15.245808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cdd94417-9a4e-495f-bd4a-55441abe6eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.255484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1080/2126] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:15.260331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd94417-9a4e-495f-bd4a-55441abe6eb5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.284184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd94417-9a4e-495f-bd4a-55441abe6eb5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.295921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.297942 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.298243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.298607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.301691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:15.303160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.306131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.312470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446c7ac-85b9-4890-a1ef-17d057cf55ec None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.316929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32949f93-e808-436a-87d3-0b2311c082e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.317774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32949f93-e808-436a-87d3-0b2311c082e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.318981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32949f93-e808-436a-87d3-0b2311c082e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.320553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32949f93-e808-436a-87d3-0b2311c082e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.332549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32949f93-e808-436a-87d3-0b2311c082e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.337220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1081/2127] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:15.337379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32949f93-e808-436a-87d3-0b2311c082e5 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.338920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.349444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1047/2128] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:15.356124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599d76-18a9-4ea8-901d-b3b80140dbb5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.362468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.363220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.363611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.363986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.366675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.369091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.448722 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1082/2129] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:15 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:15.449927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32949f93-e808-436a-87d3-0b2311c082e5 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.467405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32949f93-e808-436a-87d3-0b2311c082e5 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.513137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.515956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.537608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428a7d-bd2e-449b-8206-a6677ed8120d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.538158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428a7d-bd2e-449b-8206-a6677ed8120d admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.538501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428a7d-bd2e-449b-8206-a6677ed8120d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.538845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428a7d-bd2e-449b-8206-a6677ed8120d admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.540076 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19428a7d-bd2e-449b-8206-a6677ed8120d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.542022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19428a7d-bd2e-449b-8206-a6677ed8120d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.568161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1048/2130] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:15 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:15.571947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428a7d-bd2e-449b-8206-a6677ed8120d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.583840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428a7d-bd2e-449b-8206-a6677ed8120d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.593093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6e665-6765-43ba-acc7-636290bb845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.593711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6e665-6765-43ba-acc7-636290bb845e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.593989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6e665-6765-43ba-acc7-636290bb845e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.594264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6e665-6765-43ba-acc7-636290bb845e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.594924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6e665-6765-43ba-acc7-636290bb845e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.597290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6e665-6765-43ba-acc7-636290bb845e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.610343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:15.611875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1049/2131] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:15.617103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6e665-6765-43ba-acc7-636290bb845e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.617955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd97655-b0b1-46ea-ab0e-67db1dee8ce1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.623664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.624367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.624751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.625090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] PATH_INFO: `/v3/users/d797a449db2e4a41a5cdf0772062c955` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.626348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d797a449db2e4a41a5cdf0772062c955)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.628411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6e665-6765-43ba-acc7-636290bb845e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.628570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.637803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbac38-bf2f-4a71-b48d-236d9fe429ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.638372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbac38-bf2f-4a71-b48d-236d9fe429ae admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.638647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbac38-bf2f-4a71-b48d-236d9fe429ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.638918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbac38-bf2f-4a71-b48d-236d9fe429ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.639588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbac38-bf2f-4a71-b48d-236d9fe429ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.641210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbac38-bf2f-4a71-b48d-236d9fe429ae admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.652063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1050/2132] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:15.656133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbac38-bf2f-4a71-b48d-236d9fe429ae admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.666494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbac38-bf2f-4a71-b48d-236d9fe429ae admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.671453 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1699db-b9d6-481c-a748-ff58aac8909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.672121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1699db-b9d6-481c-a748-ff58aac8909e admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.672394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1699db-b9d6-481c-a748-ff58aac8909e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.672695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1699db-b9d6-481c-a748-ff58aac8909e admin admin] PATH_INFO: `/v3/projects/44a71170fb764e35b01742799b866561/users/adc35814ced5463fa4b5e9284b14a0e4/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.681017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd797a449db2e4a41a5cdf0772062c955'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.688922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd797a449db2e4a41a5cdf0772062c955'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.690177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1699db-b9d6-481c-a748-ff58aac8909e admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a71170fb764e35b01742799b866561, user_id=adc35814ced5463fa4b5e9284b14a0e4, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.693563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1699db-b9d6-481c-a748-ff58aac8909e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.699080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd797a449db2e4a41a5cdf0772062c955'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.699751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd797a449db2e4a41a5cdf0772062c955'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.700581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58dffa00-d4fa-4d93-bb7d-923f7fa4fa1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd797a449db2e4a41a5cdf0772062c955'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:15.711786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1051/2133] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:15 2022] PUT /identity/v3/projects/44a71170fb764e35b01742799b866561/users/adc35814ced5463fa4b5e9284b14a0e4/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:15.723233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1298d7-f145-4591-82e2-869db00a9850 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.723472 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1298d7-f145-4591-82e2-869db00a9850 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.723719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1298d7-f145-4591-82e2-869db00a9850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.730576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1083/2134] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:15 2022] DELETE /identity/v3/users/d797a449db2e4a41a5cdf0772062c955 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:15.735141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4a846-14ea-4808-a79e-6ffb4ce1b26c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.735588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4a846-14ea-4808-a79e-6ffb4ce1b26c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.735856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4a846-14ea-4808-a79e-6ffb4ce1b26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.745804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:15.756156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f7f4a846-14ea-4808-a79e-6ffb4ce1b26c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:15.756389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6f1298d7-f145-4591-82e2-869db00a9850 None None] MFA Rules not processed for user `adc35814ced5463fa4b5e9284b14a0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:15.757445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f7f4a846-14ea-4808-a79e-6ffb4ce1b26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.757642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6f1298d7-f145-4591-82e2-869db00a9850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.823589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1084/2135] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:15.825551 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce759e4e-91c8-4552-8ef2-fe4bdf075a11 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.825892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce759e4e-91c8-4552-8ef2-fe4bdf075a11 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.826173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce759e4e-91c8-4552-8ef2-fe4bdf075a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.830156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1052/2136] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:15.835160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1298d7-f145-4591-82e2-869db00a9850 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.835889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6f1298d7-f145-4591-82e2-869db00a9850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.847860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ce759e4e-91c8-4552-8ef2-fe4bdf075a11 None None] MFA Rules not processed for user `871ff162be0b43c98b531e860f6a02e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:15.849088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce759e4e-91c8-4552-8ef2-fe4bdf075a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.908343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1298d7-f145-4591-82e2-869db00a9850 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.909351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1085/2137] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:15.913484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce759e4e-91c8-4552-8ef2-fe4bdf075a11 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:15.913635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112edc2-b062-4686-a1e3-e17114cc8e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.914378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112edc2-b062-4686-a1e3-e17114cc8e09 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.914665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112edc2-b062-4686-a1e3-e17114cc8e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.914940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112edc2-b062-4686-a1e3-e17114cc8e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.915259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce759e4e-91c8-4552-8ef2-fe4bdf075a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:15.927384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112edc2-b062-4686-a1e3-e17114cc8e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.929910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112edc2-b062-4686-a1e3-e17114cc8e09 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.933470 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce759e4e-91c8-4552-8ef2-fe4bdf075a11 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:15.946789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1053/2138] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:15.952313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a67d3-8ec7-4767-aa0e-b1a5212e2ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:15.953361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a67d3-8ec7-4767-aa0e-b1a5212e2ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:15.953796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a67d3-8ec7-4767-aa0e-b1a5212e2ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:15.954249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a67d3-8ec7-4767-aa0e-b1a5212e2ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:15.955767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a67d3-8ec7-4767-aa0e-b1a5212e2ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:15.958233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a67d3-8ec7-4767-aa0e-b1a5212e2ea6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:15.987208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:16.000095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1086/2139] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:15 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:16.001077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112edc2-b062-4686-a1e3-e17114cc8e09 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.011179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112edc2-b062-4686-a1e3-e17114cc8e09 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.078079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0399a7-01c8-4a71-860e-1435ee36b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.078757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0399a7-01c8-4a71-860e-1435ee36b602 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.079198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0399a7-01c8-4a71-860e-1435ee36b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.079636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0399a7-01c8-4a71-860e-1435ee36b602 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.080319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0399a7-01c8-4a71-860e-1435ee36b602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.082623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0399a7-01c8-4a71-860e-1435ee36b602 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.106948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1054/2140] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:16 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:16.111148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a67d3-8ec7-4767-aa0e-b1a5212e2ea6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.121769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a67d3-8ec7-4767-aa0e-b1a5212e2ea6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.126407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4feeb0-0c87-4677-9718-9bee055e4009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.127025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4feeb0-0c87-4677-9718-9bee055e4009 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.127286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4feeb0-0c87-4677-9718-9bee055e4009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.127556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4feeb0-0c87-4677-9718-9bee055e4009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.128198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4feeb0-0c87-4677-9718-9bee055e4009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.129813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4feeb0-0c87-4677-9718-9bee055e4009 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.141275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1087/2141] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:16.145733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0399a7-01c8-4a71-860e-1435ee36b602 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.157515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0399a7-01c8-4a71-860e-1435ee36b602 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.164207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6175cc4-b256-42d7-b9b1-de5fe6f9abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.165026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6175cc4-b256-42d7-b9b1-de5fe6f9abc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.166385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6175cc4-b256-42d7-b9b1-de5fe6f9abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.166741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6175cc4-b256-42d7-b9b1-de5fe6f9abc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.167897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6175cc4-b256-42d7-b9b1-de5fe6f9abc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.170562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6175cc4-b256-42d7-b9b1-de5fe6f9abc0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.184475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1055/2142] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:16.187547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4feeb0-0c87-4677-9718-9bee055e4009 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.199867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4feeb0-0c87-4677-9718-9bee055e4009 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.204677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84df56-197b-40ed-aafc-6edf67b1ab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.205334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84df56-197b-40ed-aafc-6edf67b1ab4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.205631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84df56-197b-40ed-aafc-6edf67b1ab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.205887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84df56-197b-40ed-aafc-6edf67b1ab4c admin admin] PATH_INFO: `/v3/projects/71bbe1bfa84b43b19b00e6cfd7f62d50/users/1c5008993d6f43cdbb8ac0bb23fc383b/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.223804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84df56-197b-40ed-aafc-6edf67b1ab4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71bbe1bfa84b43b19b00e6cfd7f62d50, user_id=1c5008993d6f43cdbb8ac0bb23fc383b, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.227711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84df56-197b-40ed-aafc-6edf67b1ab4c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.261807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1088/2143] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:16 2022] PUT /identity/v3/projects/71bbe1bfa84b43b19b00e6cfd7f62d50/users/1c5008993d6f43cdbb8ac0bb23fc383b/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:16.273542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560b0f2-19fd-4300-b90c-f9910d8d42af None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.274082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560b0f2-19fd-4300-b90c-f9910d8d42af None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.274539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560b0f2-19fd-4300-b90c-f9910d8d42af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.316999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2560b0f2-19fd-4300-b90c-f9910d8d42af None None] MFA Rules not processed for user `1c5008993d6f43cdbb8ac0bb23fc383b`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:16.319140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2560b0f2-19fd-4300-b90c-f9910d8d42af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:16.331866 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:16.408188 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1056/2144] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:16.413933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84df56-197b-40ed-aafc-6edf67b1ab4c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.513363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560b0f2-19fd-4300-b90c-f9910d8d42af None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.515225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2560b0f2-19fd-4300-b90c-f9910d8d42af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:16.533208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84df56-197b-40ed-aafc-6edf67b1ab4c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.543253 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5b700-f7db-40a4-999a-972b4c8b9fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.543453 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560b0f2-19fd-4300-b90c-f9910d8d42af None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.544188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5b700-f7db-40a4-999a-972b4c8b9fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.544520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5b700-f7db-40a4-999a-972b4c8b9fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.544803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5b700-f7db-40a4-999a-972b4c8b9fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.545970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5b700-f7db-40a4-999a-972b4c8b9fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.548374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5b700-f7db-40a4-999a-972b4c8b9fc3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.552024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.552898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.553289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.553572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] PATH_INFO: `/v3/projects/2f5caeb5fe654297a25764b72a17762b` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.554739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5caeb5fe654297a25764b72a17762b)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.556704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.614289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1089/2145] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:16 2022] POST /identity/v3/projects => generated 387 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:16.620546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5b700-f7db-40a4-999a-972b4c8b9fc3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.667721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5caeb5fe654297a25764b72a17762b', 'request_id': 'req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.677130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5caeb5fe654297a25764b72a17762b', 'request_id': 'req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.690403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5caeb5fe654297a25764b72a17762b', 'request_id': 'req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.717149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] The token cache is being invalidate because project 2f5caeb5fe654297a25764b72a17762b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:16.719399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.721154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.727393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1057/2146] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:16 2022] DELETE /identity/v3/projects/2f5caeb5fe654297a25764b72a17762b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:16.729950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.731367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:16.733563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5b700-f7db-40a4-999a-972b4c8b9fc3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.811854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27b32-ad10-48e6-b3d5-f7d41e12bd50 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.815608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:16.817470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.818743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.819951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.820410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] PATH_INFO: `/v3/users/65276929ce6d4c9798ceb896e13844c1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.823554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] RBAC: Authorizing `identity:delete_user(user_id=65276929ce6d4c9798ceb896e13844c1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.825707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.826733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.827105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.827721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.829007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.830604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.834010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.862529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1090/2147] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:16 2022] POST /identity/v3/users => generated 300 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:16.867878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.871821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:16.887397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:16.895600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fe235-0d30-4ac5-bee3-2df891c0022b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:16.902690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65276929ce6d4c9798ceb896e13844c1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.907033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38322be8-f539-4839-9212-708d1669136a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:16.908224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38322be8-f539-4839-9212-708d1669136a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:16.909734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38322be8-f539-4839-9212-708d1669136a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:16.910344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38322be8-f539-4839-9212-708d1669136a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:16.913486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38322be8-f539-4839-9212-708d1669136a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:16.913880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65276929ce6d4c9798ceb896e13844c1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.915718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38322be8-f539-4839-9212-708d1669136a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:16.923975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65276929ce6d4c9798ceb896e13844c1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.926774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65276929ce6d4c9798ceb896e13844c1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.927569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65276929ce6d4c9798ceb896e13844c1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:16.929608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1091/2148] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:16.934554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38322be8-f539-4839-9212-708d1669136a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:16.937050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-38322be8-f539-4839-9212-708d1669136a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:16.956217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1058/2149] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:16 2022] DELETE /identity/v3/users/65276929ce6d4c9798ceb896e13844c1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:16.957242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38322be8-f539-4839-9212-708d1669136a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:17.091692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c179d-3af4-4725-a18f-3fef0fb2dc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:17.092647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c179d-3af4-4725-a18f-3fef0fb2dc61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.093032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c179d-3af4-4725-a18f-3fef0fb2dc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.093412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c179d-3af4-4725-a18f-3fef0fb2dc61 admin admin] PATH_INFO: `/v3/projects/d6b23c29768440258d753c3c94892825/users/fd8eb201bd2e49bb9a0192501682ba2d/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.112267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c179d-3af4-4725-a18f-3fef0fb2dc61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b23c29768440258d753c3c94892825, user_id=fd8eb201bd2e49bb9a0192501682ba2d, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:17.115916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c179d-3af4-4725-a18f-3fef0fb2dc61 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:17.136078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1092/2150] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:16 2022] PUT /identity/v3/projects/d6b23c29768440258d753c3c94892825/users/fd8eb201bd2e49bb9a0192501682ba2d/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:17.140105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:17.153440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30a267-786c-4912-8f49-7c073eb1864b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:17.243538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9de0d-c7ca-46c5-862a-d1024e6ce9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:17.244135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9de0d-c7ca-46c5-862a-d1024e6ce9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.244497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9de0d-c7ca-46c5-862a-d1024e6ce9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.244787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9de0d-c7ca-46c5-862a-d1024e6ce9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.245522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9de0d-c7ca-46c5-862a-d1024e6ce9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:17.247455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9de0d-c7ca-46c5-862a-d1024e6ce9f3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:17.267973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1059/2151] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:17.271980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c179d-3af4-4725-a18f-3fef0fb2dc61 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:17.288705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c179d-3af4-4725-a18f-3fef0fb2dc61 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:17.293725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:17.294407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.294687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.294960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] PATH_INFO: `/v3/projects/d6b23c29768440258d753c3c94892825/users/fd8eb201bd2e49bb9a0192501682ba2d/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.296997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b23c29768440258d753c3c94892825, user_id=fd8eb201bd2e49bb9a0192501682ba2d, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:17.301004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:17.324744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1093/2152] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:17 2022] PUT /identity/v3/projects/d6b23c29768440258d753c3c94892825/users/fd8eb201bd2e49bb9a0192501682ba2d/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:17.329494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2281b-707e-4d17-81aa-307f812e0b26 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.329786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2281b-707e-4d17-81aa-307f812e0b26 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.330065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2281b-707e-4d17-81aa-307f812e0b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.364857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-83c2281b-707e-4d17-81aa-307f812e0b26 None None] MFA Rules not processed for user `fd8eb201bd2e49bb9a0192501682ba2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:17.366705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-83c2281b-707e-4d17-81aa-307f812e0b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:17.408662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:17.410405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:17.502436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1060/2153] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:17.513178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2281b-707e-4d17-81aa-307f812e0b26 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:17.515403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-83c2281b-707e-4d17-81aa-307f812e0b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:17.561800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1444f7a-8c54-4e30-b35f-7f5416b53742 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:17.569985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:17.575117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:17.576233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.576625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.576972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] PATH_INFO: `/v3/projects/6cfadf687dc140bdad8a4dc06a21009b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.578479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfadf687dc140bdad8a4dc06a21009b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:17.582797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:17.618581 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2281b-707e-4d17-81aa-307f812e0b26 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:17.627180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cbefa-0b4a-4287-9a03-ff6bc6443135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:17.628195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cbefa-0b4a-4287-9a03-ff6bc6443135 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.628649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cbefa-0b4a-4287-9a03-ff6bc6443135 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.629101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cbefa-0b4a-4287-9a03-ff6bc6443135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.643282 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cbefa-0b4a-4287-9a03-ff6bc6443135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:17.644769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cbefa-0b4a-4287-9a03-ff6bc6443135 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:17.661044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1061/2154] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:17.668762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:17.671648 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfadf687dc140bdad8a4dc06a21009b', 'request_id': 'req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:17.689297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfadf687dc140bdad8a4dc06a21009b', 'request_id': 'req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:17.701077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfadf687dc140bdad8a4dc06a21009b', 'request_id': 'req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:17.726077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] The token cache is being invalidate because project 6cfadf687dc140bdad8a4dc06a21009b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:17.726418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:17.727166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-534a43d4-a6dd-4b9f-a8f2-cae7696c71ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:17.733380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1094/2155] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:17 2022] DELETE /identity/v3/projects/6cfadf687dc140bdad8a4dc06a21009b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:17.747959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ec42e-ae44-492a-9696-c08a9d2c034f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.747959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ec42e-ae44-492a-9696-c08a9d2c034f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.747959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ec42e-ae44-492a-9696-c08a9d2c034f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.762272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2c9ec42e-ae44-492a-9696-c08a9d2c034f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:17.764646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2c9ec42e-ae44-492a-9696-c08a9d2c034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:17.862953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1062/2156] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:17.871556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f6747-f871-436b-8896-34c777b9f605 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.871873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f6747-f871-436b-8896-34c777b9f605 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.872241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f6747-f871-436b-8896-34c777b9f605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.900364 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-295f6747-f871-436b-8896-34c777b9f605 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:17.901649 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-295f6747-f871-436b-8896-34c777b9f605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:17.912943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1095/2157] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:17.923129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ec42e-ae44-492a-9696-c08a9d2c034f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:17.940067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ec42e-ae44-492a-9696-c08a9d2c034f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:17.946521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:17.947606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:17.947998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:17.949759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:17.950545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:17.950956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:17.953852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:17.974852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1063/2158] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:17.975606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f6747-f871-436b-8896-34c777b9f605 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:18.003014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f6747-f871-436b-8896-34c777b9f605 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:18.013519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f3e30-3479-46cb-9041-8c461dc4b157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:18.014439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3e30-3479-46cb-9041-8c461dc4b157 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:18.014719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3e30-3479-46cb-9041-8c461dc4b157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:18.015018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3e30-3479-46cb-9041-8c461dc4b157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:18.015842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3e30-3479-46cb-9041-8c461dc4b157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:18.017621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3e30-3479-46cb-9041-8c461dc4b157 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:18.092190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1096/2159] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:17 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:18.096510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:18.113959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830635a-dc66-455c-a721-28de796e729c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:18.285395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047fdcc-92df-4870-b292-08151a8fb19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:18.286340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047fdcc-92df-4870-b292-08151a8fb19b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:18.287553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047fdcc-92df-4870-b292-08151a8fb19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:18.287984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047fdcc-92df-4870-b292-08151a8fb19b admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:18.288770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047fdcc-92df-4870-b292-08151a8fb19b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:18.290992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047fdcc-92df-4870-b292-08151a8fb19b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:18.340778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1064/2160] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:18 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:18.344869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f3e30-3479-46cb-9041-8c461dc4b157 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:18.361565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f3e30-3479-46cb-9041-8c461dc4b157 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:18.381468 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708a63-622e-481f-bf26-d6b13ebd3e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:18.389918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708a63-622e-481f-bf26-d6b13ebd3e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:18.389918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708a63-622e-481f-bf26-d6b13ebd3e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:18.389918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708a63-622e-481f-bf26-d6b13ebd3e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:18.389918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8708a63-622e-481f-bf26-d6b13ebd3e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:18.389918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8708a63-622e-481f-bf26-d6b13ebd3e31 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:18.416092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1097/2161] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:18.420699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047fdcc-92df-4870-b292-08151a8fb19b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:18.435278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047fdcc-92df-4870-b292-08151a8fb19b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:18.444973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:18.445739 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:18.446094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:18.447026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:18.447798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:18.450240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:18.473748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1065/2162] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:18.475498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708a63-622e-481f-bf26-d6b13ebd3e31 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:18.493956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708a63-622e-481f-bf26-d6b13ebd3e31 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:18.503271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a43e7-7610-4926-947a-9a3ab8e47662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:18.503941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a43e7-7610-4926-947a-9a3ab8e47662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:18.504229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a43e7-7610-4926-947a-9a3ab8e47662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:18.505131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a43e7-7610-4926-947a-9a3ab8e47662 admin admin] PATH_INFO: `/v3/projects/8322378104634943b7be22cd0719d561/users/112011e519dc4f2eba411b446bf3cd20/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:18.535881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a43e7-7610-4926-947a-9a3ab8e47662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8322378104634943b7be22cd0719d561, user_id=112011e519dc4f2eba411b446bf3cd20, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:18.539678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a43e7-7610-4926-947a-9a3ab8e47662 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:18.561402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1098/2163] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:18 2022] PUT /identity/v3/projects/8322378104634943b7be22cd0719d561/users/112011e519dc4f2eba411b446bf3cd20/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:18.566237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:18.567987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:18.578236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aec54f-7e18-44a1-a29d-1e0a93ba3f7f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:18.578523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aec54f-7e18-44a1-a29d-1e0a93ba3f7f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:18.579896 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aec54f-7e18-44a1-a29d-1e0a93ba3f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:18.617243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f4aec54f-7e18-44a1-a29d-1e0a93ba3f7f None None] MFA Rules not processed for user `112011e519dc4f2eba411b446bf3cd20`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:18.618561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f4aec54f-7e18-44a1-a29d-1e0a93ba3f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:18.627797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:18.648386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552846-17e5-4f95-a1df-ee098c0b270e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:18.715745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:18.716367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:18.716696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:18.716971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:18.718791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:18.778209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1099/2164] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:18.781019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aec54f-7e18-44a1-a29d-1e0a93ba3f7f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:18.851816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:18.855697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:18.879334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1066/2165] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:18.923219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aec54f-7e18-44a1-a29d-1e0a93ba3f7f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:18.934122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0f20-da0b-4420-91a1-41fd3e57f8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:18.935024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0f20-da0b-4420-91a1-41fd3e57f8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:18.935498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0f20-da0b-4420-91a1-41fd3e57f8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:18.936011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d0f20-da0b-4420-91a1-41fd3e57f8a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:18.937189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d0f20-da0b-4420-91a1-41fd3e57f8a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:18.939305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d0f20-da0b-4420-91a1-41fd3e57f8a3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:18.948703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:19.002597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1100/2166] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:18 2022] POST /identity/v3/projects => generated 387 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:19.008304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:19.029636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e739812-bb2b-4d12-9aad-adf1c3010703 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:19.150798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d4c45-3d11-468b-84a7-e3b80de77152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:19.151626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4c45-3d11-468b-84a7-e3b80de77152 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:19.152045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4c45-3d11-468b-84a7-e3b80de77152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:19.152565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4c45-3d11-468b-84a7-e3b80de77152 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:19.153411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d4c45-3d11-468b-84a7-e3b80de77152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:19.157871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d4c45-3d11-468b-84a7-e3b80de77152 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:19.192910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1067/2167] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:19 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:19.195299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0f20-da0b-4420-91a1-41fd3e57f8a3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:19.210597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d0f20-da0b-4420-91a1-41fd3e57f8a3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:19.216040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6ab71-7044-463a-a365-1e408ff7db15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:19.216857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6ab71-7044-463a-a365-1e408ff7db15 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:19.217276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6ab71-7044-463a-a365-1e408ff7db15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:19.217525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6ab71-7044-463a-a365-1e408ff7db15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:19.218286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6ab71-7044-463a-a365-1e408ff7db15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:19.220323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6ab71-7044-463a-a365-1e408ff7db15 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:19.240534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d4c45-3d11-468b-84a7-e3b80de77152 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:19.241028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1101/2168] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:19.253308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d4c45-3d11-468b-84a7-e3b80de77152 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:19.274578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b60e82-ed93-43c1-88cd-b1ba172ecb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:19.275490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b60e82-ed93-43c1-88cd-b1ba172ecb61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:19.275773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b60e82-ed93-43c1-88cd-b1ba172ecb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:19.276049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b60e82-ed93-43c1-88cd-b1ba172ecb61 admin admin] PATH_INFO: `/v3/projects/d676fa3d5a3542a8ba51e7a18105e75f/users/84aea21dd0674d3f8ee37eebd13027a9/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:19.324032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b60e82-ed93-43c1-88cd-b1ba172ecb61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d676fa3d5a3542a8ba51e7a18105e75f, user_id=84aea21dd0674d3f8ee37eebd13027a9, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:19.332095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b60e82-ed93-43c1-88cd-b1ba172ecb61 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:19.364109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1068/2169] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:19 2022] PUT /identity/v3/projects/d676fa3d5a3542a8ba51e7a18105e75f/users/84aea21dd0674d3f8ee37eebd13027a9/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:19.366139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6ab71-7044-463a-a365-1e408ff7db15 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:19.384886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6ab71-7044-463a-a365-1e408ff7db15 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:19.512819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff56659-ca0e-476a-81cf-d60e7e9c80b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:19.514535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff56659-ca0e-476a-81cf-d60e7e9c80b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:19.514996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff56659-ca0e-476a-81cf-d60e7e9c80b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:19.515450 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff56659-ca0e-476a-81cf-d60e7e9c80b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:19.516718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff56659-ca0e-476a-81cf-d60e7e9c80b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:19.518854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff56659-ca0e-476a-81cf-d60e7e9c80b9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:19.541341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1102/2170] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:19.545024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b60e82-ed93-43c1-88cd-b1ba172ecb61 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:19.559843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b60e82-ed93-43c1-88cd-b1ba172ecb61 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:19.566892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:19.569504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:19.569960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:19.570387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] PATH_INFO: `/v3/projects/d676fa3d5a3542a8ba51e7a18105e75f/users/84aea21dd0674d3f8ee37eebd13027a9/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:19.573525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d676fa3d5a3542a8ba51e7a18105e75f, user_id=84aea21dd0674d3f8ee37eebd13027a9, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:19.577083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:19.590616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1069/2171] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:19 2022] PUT /identity/v3/projects/d676fa3d5a3542a8ba51e7a18105e75f/users/84aea21dd0674d3f8ee37eebd13027a9/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:19.601957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8d9b3-e5ce-4681-817f-fe2971dd46d1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:19.602316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8d9b3-e5ce-4681-817f-fe2971dd46d1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:19.602523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8d9b3-e5ce-4681-817f-fe2971dd46d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:19.641767 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-75c8d9b3-e5ce-4681-817f-fe2971dd46d1 None None] MFA Rules not processed for user `84aea21dd0674d3f8ee37eebd13027a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:19.643043 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-75c8d9b3-e5ce-4681-817f-fe2971dd46d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:19.748588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1103/2172] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:19.760752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:19.762508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:19.770254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:19.868903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2f04-58c4-47ea-af69-b6b26ed3e9ca admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:19.872820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f8553-6333-457d-8f70-fbacfaadc3d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:19.873445 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f8553-6333-457d-8f70-fbacfaadc3d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:19.873725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f8553-6333-457d-8f70-fbacfaadc3d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:19.874007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f8553-6333-457d-8f70-fbacfaadc3d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:19.884648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f8553-6333-457d-8f70-fbacfaadc3d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:19.886431 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f8553-6333-457d-8f70-fbacfaadc3d8 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:19.898706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1070/2173] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:20.819138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8d9b3-e5ce-4681-817f-fe2971dd46d1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:20.902374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:20.907307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8d9b3-e5ce-4681-817f-fe2971dd46d1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:20.913806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:20.914444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:20.914750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:20.915026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] PATH_INFO: `/v3/users/112011e519dc4f2eba411b446bf3cd20` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:20.916154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=112011e519dc4f2eba411b446bf3cd20)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:20.918206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:20.966693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112011e519dc4f2eba411b446bf3cd20'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:20.974055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112011e519dc4f2eba411b446bf3cd20'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:20.982025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112011e519dc4f2eba411b446bf3cd20'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:20.982729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112011e519dc4f2eba411b446bf3cd20'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:20.983344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112011e519dc4f2eba411b446bf3cd20'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.001643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1104/2174] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:20 2022] DELETE /identity/v3/users/112011e519dc4f2eba411b446bf3cd20 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:21.006326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb32b6-cfe9-4439-9473-a04bc40e0bdd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:21.006808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb32b6-cfe9-4439-9473-a04bc40e0bdd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:21.007042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb32b6-cfe9-4439-9473-a04bc40e0bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:21.027095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a0fb32b6-cfe9-4439-9473-a04bc40e0bdd None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:21.028295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a0fb32b6-cfe9-4439-9473-a04bc40e0bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:21.091648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1071/2175] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:21.102772 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:21.103640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:21.179602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed801e-d216-410d-9b12-cf3870dfa05c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:21.183619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e52f30-4799-48a1-931b-3a09887dc635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:21.184432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e52f30-4799-48a1-931b-3a09887dc635 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:21.184739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e52f30-4799-48a1-931b-3a09887dc635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:21.185010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e52f30-4799-48a1-931b-3a09887dc635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:21.196002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e52f30-4799-48a1-931b-3a09887dc635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:21.197863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e52f30-4799-48a1-931b-3a09887dc635 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:21.209377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1105/2176] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:21.550378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb32b6-cfe9-4439-9473-a04bc40e0bdd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:21.553761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a0fb32b6-cfe9-4439-9473-a04bc40e0bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:21.572867 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb32b6-cfe9-4439-9473-a04bc40e0bdd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:21.577887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:21.578530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:21.579163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:21.579558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] PATH_INFO: `/v3/projects/8322378104634943b7be22cd0719d561` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:21.580947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8322378104634943b7be22cd0719d561)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:21.584725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:21.639152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8322378104634943b7be22cd0719d561', 'request_id': 'req-85d76dce-6885-42ba-8ce5-3d3893732b49'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.645662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8322378104634943b7be22cd0719d561', 'request_id': 'req-85d76dce-6885-42ba-8ce5-3d3893732b49'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.658709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8322378104634943b7be22cd0719d561', 'request_id': 'req-85d76dce-6885-42ba-8ce5-3d3893732b49'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.673406 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] The token cache is being invalidate because project 8322378104634943b7be22cd0719d561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:21.674613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.674727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.677103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1072/2177] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:21 2022] DELETE /identity/v3/projects/8322378104634943b7be22cd0719d561 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:21.681832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e52f30-4799-48a1-931b-3a09887dc635 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:21.691516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e52f30-4799-48a1-931b-3a09887dc635 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:21.766997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:21.768124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:21.768614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:21.769102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] PATH_INFO: `/v3/users/84aea21dd0674d3f8ee37eebd13027a9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:21.775057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84aea21dd0674d3f8ee37eebd13027a9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:21.777943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:21.830769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:21.887812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84aea21dd0674d3f8ee37eebd13027a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.915603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84aea21dd0674d3f8ee37eebd13027a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.958109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84aea21dd0674d3f8ee37eebd13027a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.966519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84aea21dd0674d3f8ee37eebd13027a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.967175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97ce6ca7-e8e2-4ff4-b024-d9656f71b9d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84aea21dd0674d3f8ee37eebd13027a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:21.998144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1106/2178] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:21 2022] DELETE /identity/v3/users/84aea21dd0674d3f8ee37eebd13027a9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:22.347259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:22.348968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:22.367067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d76dce-6885-42ba-8ce5-3d3893732b49 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:22.428202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:22.456248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:22.456922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:22.457202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:22.457475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] PATH_INFO: `/v3/projects/d676fa3d5a3542a8ba51e7a18105e75f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:22.459178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d676fa3d5a3542a8ba51e7a18105e75f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:22.460891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:22.507229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd676fa3d5a3542a8ba51e7a18105e75f', 'request_id': 'req-20282850-e829-4daa-a66e-f8aa11dc0ca3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.513028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd676fa3d5a3542a8ba51e7a18105e75f', 'request_id': 'req-20282850-e829-4daa-a66e-f8aa11dc0ca3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.520080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd676fa3d5a3542a8ba51e7a18105e75f', 'request_id': 'req-20282850-e829-4daa-a66e-f8aa11dc0ca3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.531021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] The token cache is being invalidate because project d676fa3d5a3542a8ba51e7a18105e75f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:22.531306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.531950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-20282850-e829-4daa-a66e-f8aa11dc0ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.533447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1073/2179] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:22 2022] DELETE /identity/v3/projects/d676fa3d5a3542a8ba51e7a18105e75f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:22.538712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cba21-bf99-4d45-b2ab-8f13ad5d72fd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:22.539070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cba21-bf99-4d45-b2ab-8f13ad5d72fd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:22.539413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cba21-bf99-4d45-b2ab-8f13ad5d72fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:22.560729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fe4cba21-bf99-4d45-b2ab-8f13ad5d72fd None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:22.561956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fe4cba21-bf99-4d45-b2ab-8f13ad5d72fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:22.627631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1107/2180] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:22.636672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad6fcf-e81e-42d2-badd-8ca55f24e559 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:22.636984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad6fcf-e81e-42d2-badd-8ca55f24e559 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:22.637264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad6fcf-e81e-42d2-badd-8ca55f24e559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:22.649033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05eb0-a134-4f19-9a8e-5067f67bac72 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:22.649336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05eb0-a134-4f19-9a8e-5067f67bac72 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:22.649594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05eb0-a134-4f19-9a8e-5067f67bac72 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:22.651233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1108/2181] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:15:22.659744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c2ad6fcf-e81e-42d2-badd-8ca55f24e559 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:22.660950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c2ad6fcf-e81e-42d2-badd-8ca55f24e559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:22.668973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1074/2182] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:22.673380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa05eb0-a134-4f19-9a8e-5067f67bac72 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:22.676230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b99b-4287-42f5-99d4-55a0dc737b1f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:22.676541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b99b-4287-42f5-99d4-55a0dc737b1f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:22.678804 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b99b-4287-42f5-99d4-55a0dc737b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:22.689214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa05eb0-a134-4f19-9a8e-5067f67bac72 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:22.698085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:22.698725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:22.699010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:22.699280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:22.699955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:22.700416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:22.702198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:22.726857 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1109/2183] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:22.729540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:22.730099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b1d7b99b-4287-42f5-99d4-55a0dc737b1f None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:22.731458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b1d7b99b-4287-42f5-99d4-55a0dc737b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:22.731742 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:22.746717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e5e9e-06d8-4e82-a838-bf0c3a230bdb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:22.754801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:22.754801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:22.754801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:22.755166 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] PATH_INFO: `/v3/users/5464533ccfb74094bf613bcd51c55ff1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:22.761929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5464533ccfb74094bf613bcd51c55ff1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:22.764627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:22.817273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5464533ccfb74094bf613bcd51c55ff1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.821988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1075/2184] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:22.822679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b99b-4287-42f5-99d4-55a0dc737b1f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:22.825434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5464533ccfb74094bf613bcd51c55ff1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.836632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5464533ccfb74094bf613bcd51c55ff1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.837362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b99b-4287-42f5-99d4-55a0dc737b1f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:22.837447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5464533ccfb74094bf613bcd51c55ff1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.837933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5464533ccfb74094bf613bcd51c55ff1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:22.853047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:22.858299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:22.859479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:22.859816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:22.860211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:22.860617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1110/2185] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:22 2022] DELETE /identity/v3/users/5464533ccfb74094bf613bcd51c55ff1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:22.862282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:22.864831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:22.868313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:22.871125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:22.939319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1076/2186] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:22 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:22.944727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:22.947349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:23.019731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53436d9-452e-44aa-b1b6-addf00ef44a5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:23.027632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:23.093561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677c4cf-0ac1-419c-80bc-c280c68e8019 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:23.110071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:23.111403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.111694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.112044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.114858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d41c1-7f33-4e9c-a0a8-15d1b84ca002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:23.115017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:23.115870 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d41c1-7f33-4e9c-a0a8-15d1b84ca002 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.116357 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d41c1-7f33-4e9c-a0a8-15d1b84ca002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.116861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d41c1-7f33-4e9c-a0a8-15d1b84ca002 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.118782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d41c1-7f33-4e9c-a0a8-15d1b84ca002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:23.121384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d41c1-7f33-4e9c-a0a8-15d1b84ca002 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:23.132909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:23.136740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:23.168881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1077/2187] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:22 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:23.170879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d41c1-7f33-4e9c-a0a8-15d1b84ca002 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:23.183702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1111/2188] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:23.198586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d41c1-7f33-4e9c-a0a8-15d1b84ca002 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:23.210919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566f6a-4c8b-413a-95d6-83310f2d32fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:23.212227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566f6a-4c8b-413a-95d6-83310f2d32fc admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.215066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566f6a-4c8b-413a-95d6-83310f2d32fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.215409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566f6a-4c8b-413a-95d6-83310f2d32fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.216295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88566f6a-4c8b-413a-95d6-83310f2d32fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:23.218059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88566f6a-4c8b-413a-95d6-83310f2d32fc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:23.248377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1078/2189] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:23.253658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:23.320359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8213f3-a166-4adc-8635-cd537a25eef9 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:23.330707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27aa6a-96fc-46d3-a148-8fc5dac99c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:23.331459 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27aa6a-96fc-46d3-a148-8fc5dac99c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.331841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27aa6a-96fc-46d3-a148-8fc5dac99c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.332502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27aa6a-96fc-46d3-a148-8fc5dac99c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.333262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27aa6a-96fc-46d3-a148-8fc5dac99c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:23.344766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27aa6a-96fc-46d3-a148-8fc5dac99c20 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:23.362822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1112/2190] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:23.367209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566f6a-4c8b-413a-95d6-83310f2d32fc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:23.378977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566f6a-4c8b-413a-95d6-83310f2d32fc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:23.399242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf481e2b-f118-4302-a76e-2650dbe4c526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:23.400206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf481e2b-f118-4302-a76e-2650dbe4c526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.400720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf481e2b-f118-4302-a76e-2650dbe4c526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.401179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf481e2b-f118-4302-a76e-2650dbe4c526 admin admin] PATH_INFO: `/v3/projects/0fb17505eab145bca878b048ab09b159/users/b23c056fb90b43049f358b4c9058c987/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.419247 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf481e2b-f118-4302-a76e-2650dbe4c526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb17505eab145bca878b048ab09b159, user_id=b23c056fb90b43049f358b4c9058c987, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:23.422050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf481e2b-f118-4302-a76e-2650dbe4c526 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:23.438656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1079/2191] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:23 2022] PUT /identity/v3/projects/0fb17505eab145bca878b048ab09b159/users/b23c056fb90b43049f358b4c9058c987/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:23.455684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c8e10-75c2-42d4-83f5-5d37db37ab8b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.456082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c8e10-75c2-42d4-83f5-5d37db37ab8b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.456463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c8e10-75c2-42d4-83f5-5d37db37ab8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.510624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-027c8e10-75c2-42d4-83f5-5d37db37ab8b None None] MFA Rules not processed for user `b23c056fb90b43049f358b4c9058c987`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:23.512642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-027c8e10-75c2-42d4-83f5-5d37db37ab8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:23.602003 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1113/2192] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:23.606696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf481e2b-f118-4302-a76e-2650dbe4c526 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:23.633775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c8e10-75c2-42d4-83f5-5d37db37ab8b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:23.635630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-027c8e10-75c2-42d4-83f5-5d37db37ab8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:23.653225 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:23.709847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf481e2b-f118-4302-a76e-2650dbe4c526 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:23.715202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b2184-0462-45eb-9750-712494e5e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:23.715844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2184-0462-45eb-9750-712494e5e6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.716125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2184-0462-45eb-9750-712494e5e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.716397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2184-0462-45eb-9750-712494e5e6f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.717192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b2184-0462-45eb-9750-712494e5e6f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:23.719089 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b2184-0462-45eb-9750-712494e5e6f7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:23.728655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c8e10-75c2-42d4-83f5-5d37db37ab8b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:23.733745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:23.734581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.735034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.735519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] PATH_INFO: `/v3/projects/df78df5c522b42b1b40aebf2502866e7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.736911 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=df78df5c522b42b1b40aebf2502866e7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:23.738962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:23.751579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1080/2193] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:23 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:23.756133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b2184-0462-45eb-9750-712494e5e6f7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:23.806523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df78df5c522b42b1b40aebf2502866e7', 'request_id': 'req-80181185-0570-44c6-9e16-20fe61a51cca'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:23.814299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df78df5c522b42b1b40aebf2502866e7', 'request_id': 'req-80181185-0570-44c6-9e16-20fe61a51cca'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:23.822995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df78df5c522b42b1b40aebf2502866e7', 'request_id': 'req-80181185-0570-44c6-9e16-20fe61a51cca'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:23.831491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b2184-0462-45eb-9750-712494e5e6f7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:23.836535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] The token cache is being invalidate because project df78df5c522b42b1b40aebf2502866e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:23.837011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:23.838123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-80181185-0570-44c6-9e16-20fe61a51cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:23.840094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1114/2194] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:23 2022] DELETE /identity/v3/projects/df78df5c522b42b1b40aebf2502866e7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:23.846305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0feaf8-d7df-4fcb-8422-b1de886cfd93 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.846866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0feaf8-d7df-4fcb-8422-b1de886cfd93 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.847381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0feaf8-d7df-4fcb-8422-b1de886cfd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.852808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:23.874470 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dd0feaf8-d7df-4fcb-8422-b1de886cfd93 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:23.875868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dd0feaf8-d7df-4fcb-8422-b1de886cfd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:23.923348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:23.924408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:23.934210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:23.934492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:23.935167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:23.937341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:23.979116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1081/2195] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:23 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:23.983033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:23.983955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1115/2196] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:23.985774 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:24.001746 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b953786-eaae-4feb-a65f-e19ba22da8a6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.002068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b953786-eaae-4feb-a65f-e19ba22da8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.002370 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b953786-eaae-4feb-a65f-e19ba22da8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.008257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55d25-ce10-4d8a-b5b0-96c63567c2f4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.014246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ddd19-b281-465f-aac2-03b0e2e4c7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.014864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ddd19-b281-465f-aac2-03b0e2e4c7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.015157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ddd19-b281-465f-aac2-03b0e2e4c7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.015422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ddd19-b281-465f-aac2-03b0e2e4c7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.016202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ddd19-b281-465f-aac2-03b0e2e4c7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.018462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ddd19-b281-465f-aac2-03b0e2e4c7c2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.032598 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1082/2197] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:24.036065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ddd19-b281-465f-aac2-03b0e2e4c7c2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.048302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7b953786-eaae-4feb-a65f-e19ba22da8a6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:24.050156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ddd19-b281-465f-aac2-03b0e2e4c7c2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.055348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7b953786-eaae-4feb-a65f-e19ba22da8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:24.058997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c0231-14f6-48c1-9029-8fcd0e78b2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.059541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c0231-14f6-48c1-9029-8fcd0e78b2a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.059819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c0231-14f6-48c1-9029-8fcd0e78b2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.060094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c0231-14f6-48c1-9029-8fcd0e78b2a4 admin admin] PATH_INFO: `/v3/projects/e419640131024b4daf3f33189fbde7d9/users/87b9c6fdf7b1431e881376af8e1369ee/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.080485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c0231-14f6-48c1-9029-8fcd0e78b2a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e419640131024b4daf3f33189fbde7d9, user_id=87b9c6fdf7b1431e881376af8e1369ee, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.084936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c0231-14f6-48c1-9029-8fcd0e78b2a4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.097264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1116/2198] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:24.108464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b953786-eaae-4feb-a65f-e19ba22da8a6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.122307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1083/2199] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:24 2022] PUT /identity/v3/projects/e419640131024b4daf3f33189fbde7d9/users/87b9c6fdf7b1431e881376af8e1369ee/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:24.126792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c0231-14f6-48c1-9029-8fcd0e78b2a4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.129464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b953786-eaae-4feb-a65f-e19ba22da8a6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.187032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:24.237480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c0231-14f6-48c1-9029-8fcd0e78b2a4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.244661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a6216-075b-456d-a89b-74afec45e06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.245376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6216-075b-456d-a89b-74afec45e06a admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.245849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6216-075b-456d-a89b-74afec45e06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.246617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6216-075b-456d-a89b-74afec45e06a admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.247359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a6216-075b-456d-a89b-74afec45e06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.249508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a6216-075b-456d-a89b-74afec45e06a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.268171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.270714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.271689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.272004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.274813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:24.275129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.278001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.278120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1084/2200] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:24.280199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a6216-075b-456d-a89b-74afec45e06a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.306711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1117/2201] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:24.308744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a6216-075b-456d-a89b-74afec45e06a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.312979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.319343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f36b-4ce7-431d-9045-043394c4a662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.321075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f36b-4ce7-431d-9045-043394c4a662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.321528 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f36b-4ce7-431d-9045-043394c4a662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.321812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f36b-4ce7-431d-9045-043394c4a662 admin admin] PATH_INFO: `/v3/projects/e419640131024b4daf3f33189fbde7d9/users/87b9c6fdf7b1431e881376af8e1369ee/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.324384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0f36b-4ce7-431d-9045-043394c4a662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e419640131024b4daf3f33189fbde7d9, user_id=87b9c6fdf7b1431e881376af8e1369ee, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.327428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0f36b-4ce7-431d-9045-043394c4a662 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.328470 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847c6e-6dd9-440d-8bfa-7e09086ee7a1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.335929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb33c9-db5c-4ef4-a5b4-f86f7529ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.336997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb33c9-db5c-4ef4-a5b4-f86f7529ddbf admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.337541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb33c9-db5c-4ef4-a5b4-f86f7529ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.338162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb33c9-db5c-4ef4-a5b4-f86f7529ddbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.339191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb33c9-db5c-4ef4-a5b4-f86f7529ddbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.342054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb33c9-db5c-4ef4-a5b4-f86f7529ddbf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.343091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1085/2202] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:24 2022] PUT /identity/v3/projects/e419640131024b4daf3f33189fbde7d9/users/87b9c6fdf7b1431e881376af8e1369ee/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:24.363960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6cdf9-536d-48f8-bcb8-ad220fad3240 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.364466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6cdf9-536d-48f8-bcb8-ad220fad3240 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.364991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6cdf9-536d-48f8-bcb8-ad220fad3240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.408914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1118/2203] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:24 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:24.415737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb33c9-db5c-4ef4-a5b4-f86f7529ddbf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.435998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-82f6cdf9-536d-48f8-bcb8-ad220fad3240 None None] MFA Rules not processed for user `87b9c6fdf7b1431e881376af8e1369ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:24.438083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-82f6cdf9-536d-48f8-bcb8-ad220fad3240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:24.512284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb33c9-db5c-4ef4-a5b4-f86f7529ddbf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.517530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58227da-b9dd-45ae-b4ae-c9679a73f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.518149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58227da-b9dd-45ae-b4ae-c9679a73f413 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.519111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58227da-b9dd-45ae-b4ae-c9679a73f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.519396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58227da-b9dd-45ae-b4ae-c9679a73f413 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.520486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58227da-b9dd-45ae-b4ae-c9679a73f413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.523424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58227da-b9dd-45ae-b4ae-c9679a73f413 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.524101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:24.557212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1119/2204] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:24 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:24.562325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58227da-b9dd-45ae-b4ae-c9679a73f413 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.569881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1086/2205] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:24.579479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58227da-b9dd-45ae-b4ae-c9679a73f413 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.582852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6cdf9-536d-48f8-bcb8-ad220fad3240 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.584848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-82f6cdf9-536d-48f8-bcb8-ad220fad3240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:24.585609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f0c31-fb22-4153-8088-2bfb39db2f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.586431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f0c31-fb22-4153-8088-2bfb39db2f1c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.586771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f0c31-fb22-4153-8088-2bfb39db2f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.587132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f0c31-fb22-4153-8088-2bfb39db2f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.588092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f0c31-fb22-4153-8088-2bfb39db2f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.589965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f0c31-fb22-4153-8088-2bfb39db2f1c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.612473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1120/2206] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:24.613935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f0c31-fb22-4153-8088-2bfb39db2f1c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.624063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f0c31-fb22-4153-8088-2bfb39db2f1c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.628764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8283f-c240-428c-a91a-a3fe29f9a8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.629428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8283f-c240-428c-a91a-a3fe29f9a8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.629773 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8283f-c240-428c-a91a-a3fe29f9a8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.630120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8283f-c240-428c-a91a-a3fe29f9a8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.631374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8283f-c240-428c-a91a-a3fe29f9a8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.633156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8283f-c240-428c-a91a-a3fe29f9a8b4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.643983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1121/2207] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:24.648843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8283f-c240-428c-a91a-a3fe29f9a8b4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.657408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6cdf9-536d-48f8-bcb8-ad220fad3240 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.658959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8283f-c240-428c-a91a-a3fe29f9a8b4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.661098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2de27c-cf67-4e47-98a9-9cec9bbee73f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.661767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2de27c-cf67-4e47-98a9-9cec9bbee73f service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.662246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2de27c-cf67-4e47-98a9-9cec9bbee73f service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.662588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2de27c-cf67-4e47-98a9-9cec9bbee73f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.664134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14530-b70a-4a6d-989a-cb0a50ea5a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.664811 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14530-b70a-4a6d-989a-cb0a50ea5a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.665112 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14530-b70a-4a6d-989a-cb0a50ea5a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.665384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14530-b70a-4a6d-989a-cb0a50ea5a10 admin admin] PATH_INFO: `/v3/projects/56cc04948356488f8722a735954fd650/users/05147fb1762748ee9866ccc5bff4d514/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.672395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2de27c-cf67-4e47-98a9-9cec9bbee73f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.673882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2de27c-cf67-4e47-98a9-9cec9bbee73f service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.683683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14530-b70a-4a6d-989a-cb0a50ea5a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56cc04948356488f8722a735954fd650, user_id=05147fb1762748ee9866ccc5bff4d514, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.685464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1087/2208] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:24.690194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14530-b70a-4a6d-989a-cb0a50ea5a10 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.707479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1122/2209] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:24 2022] PUT /identity/v3/projects/56cc04948356488f8722a735954fd650/users/05147fb1762748ee9866ccc5bff4d514/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:24.712940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c4ec-55e6-43b8-8940-9f807df21c49 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.713553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c4ec-55e6-43b8-8940-9f807df21c49 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.713824 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c4ec-55e6-43b8-8940-9f807df21c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.735956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9046c4ec-55e6-43b8-8940-9f807df21c49 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:24.738031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9046c4ec-55e6-43b8-8940-9f807df21c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:24.778327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14530-b70a-4a6d-989a-cb0a50ea5a10 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.824672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1088/2210] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:24.830648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c4ec-55e6-43b8-8940-9f807df21c49 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:24.832299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9046c4ec-55e6-43b8-8940-9f807df21c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:24.846500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:24.851158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14530-b70a-4a6d-989a-cb0a50ea5a10 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.855975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c671f4-14c3-4f2e-ac75-3b43583fa31e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.856752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c671f4-14c3-4f2e-ac75-3b43583fa31e service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.857022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c671f4-14c3-4f2e-ac75-3b43583fa31e service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.857484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c671f4-14c3-4f2e-ac75-3b43583fa31e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.867001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a5c671f4-14c3-4f2e-ac75-3b43583fa31e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:24.899211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c4ec-55e6-43b8-8940-9f807df21c49 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:24.903338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bf940-6fad-4a31-9123-ccf0e8783d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:24.903954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bf940-6fad-4a31-9123-ccf0e8783d5a service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:24.904281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bf940-6fad-4a31-9123-ccf0e8783d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:24.904592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bf940-6fad-4a31-9123-ccf0e8783d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:24.915618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bf940-6fad-4a31-9123-ccf0e8783d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.918544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bf940-6fad-4a31-9123-ccf0e8783d5a service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.933638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1089/2211] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:24.946200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c671f4-14c3-4f2e-ac75-3b43583fa31e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:24.947888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c671f4-14c3-4f2e-ac75-3b43583fa31e service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:24.960165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1123/2212] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:25.269206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bf940-6fad-4a31-9123-ccf0e8783d5a service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:25.279885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bf940-6fad-4a31-9123-ccf0e8783d5a service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:25.284689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:25.285312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:25.285590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:25.285860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] PATH_INFO: `/v3/users/b23c056fb90b43049f358b4c9058c987` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:25.287274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23c056fb90b43049f358b4c9058c987)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:25.289227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:25.318606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:25.374065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23c056fb90b43049f358b4c9058c987'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:25.386288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23c056fb90b43049f358b4c9058c987'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:25.405204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23c056fb90b43049f358b4c9058c987'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:25.406546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23c056fb90b43049f358b4c9058c987'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:25.407365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23c056fb90b43049f358b4c9058c987'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:25.437625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1090/2213] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:25 2022] DELETE /identity/v3/users/b23c056fb90b43049f358b4c9058c987 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:25.442634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7a77-b8bd-4dd6-b95b-12278fc5a3ba None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:25.443336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7a77-b8bd-4dd6-b95b-12278fc5a3ba None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:25.444016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7a77-b8bd-4dd6-b95b-12278fc5a3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:25.496771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7ccb7a77-b8bd-4dd6-b95b-12278fc5a3ba None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:25.496771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ccb7a77-b8bd-4dd6-b95b-12278fc5a3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:25.636821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1124/2214] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:25.654602 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:25.656559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:25.745990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894f124-10f4-4f57-969a-83c644478159 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:25.750275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:25.751573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:25.752101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:25.752614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:25.767435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:25.770172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:25.784315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1091/2215] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:25.817283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7a77-b8bd-4dd6-b95b-12278fc5a3ba None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:25.819470 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ccb7a77-b8bd-4dd6-b95b-12278fc5a3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:25.898610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7a77-b8bd-4dd6-b95b-12278fc5a3ba None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:25.903753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:25.904806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:25.905284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:25.905768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:25.908494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:26.051371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:26.051371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:26.053724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:26.067055 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1125/2216] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:27.105363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:27.107082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:27.131185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27c369-e9a6-4838-82f3-dc973dcd9a19 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:27.148893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:27.149485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:27.149836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:27.150325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] PATH_INFO: `/v3/projects/0fb17505eab145bca878b048ab09b159` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:27.157930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb17505eab145bca878b048ab09b159)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:27.160629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:27.269273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb17505eab145bca878b048ab09b159', 'request_id': 'req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.269273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb17505eab145bca878b048ab09b159', 'request_id': 'req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.277576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb17505eab145bca878b048ab09b159', 'request_id': 'req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.305192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] The token cache is being invalidate because project 0fb17505eab145bca878b048ab09b159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:27.305652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.307936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.311364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1092/2217] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:27 2022] DELETE /identity/v3/projects/0fb17505eab145bca878b048ab09b159 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:27.315752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:27.340571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1872c22-946b-4293-9a5b-572cc9e893bb service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:27.499287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:27.499939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:27.500218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:27.500515 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] PATH_INFO: `/v3/users/87b9c6fdf7b1431e881376af8e1369ee` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:27.501730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b9c6fdf7b1431e881376af8e1369ee)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:27.503885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:27.612399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b9c6fdf7b1431e881376af8e1369ee'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.628239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b9c6fdf7b1431e881376af8e1369ee'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.642837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b9c6fdf7b1431e881376af8e1369ee'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.643506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b9c6fdf7b1431e881376af8e1369ee'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.644158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d3e6c69e-209e-45f1-9201-eff8cdad64ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b9c6fdf7b1431e881376af8e1369ee'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:27.706717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1126/2218] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:27 2022] DELETE /identity/v3/users/87b9c6fdf7b1431e881376af8e1369ee => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:28.431650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:28.433658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:28.446819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cc8f1-be6b-49ff-91cd-ee6aa2c6d9f3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:28.466977 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:28.527893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:28.528745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:28.529270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:28.529642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] PATH_INFO: `/v3/projects/e419640131024b4daf3f33189fbde7d9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:28.531659 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e419640131024b4daf3f33189fbde7d9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:28.534108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:28.616367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e419640131024b4daf3f33189fbde7d9', 'request_id': 'req-c59b2294-e800-4cd0-8878-c1ea21c61d5c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:28.627228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e419640131024b4daf3f33189fbde7d9', 'request_id': 'req-c59b2294-e800-4cd0-8878-c1ea21c61d5c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:28.651450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e419640131024b4daf3f33189fbde7d9', 'request_id': 'req-c59b2294-e800-4cd0-8878-c1ea21c61d5c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:28.691080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] The token cache is being invalidate because project e419640131024b4daf3f33189fbde7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:28.691574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:28.692251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c59b2294-e800-4cd0-8878-c1ea21c61d5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:28.698776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1093/2219] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:28 2022] DELETE /identity/v3/projects/e419640131024b4daf3f33189fbde7d9 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:28.700745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb957-c0db-4230-b4b7-8ab9983d2e2b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:28.701212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb957-c0db-4230-b4b7-8ab9983d2e2b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:28.701747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb957-c0db-4230-b4b7-8ab9983d2e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:28.730626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-787cb957-c0db-4230-b4b7-8ab9983d2e2b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:28.732367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-787cb957-c0db-4230-b4b7-8ab9983d2e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:28.860689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1127/2220] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:28.870119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424e58a-d668-4f29-b2a7-0c9229a0dc6d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:28.870746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424e58a-d668-4f29-b2a7-0c9229a0dc6d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:28.871192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424e58a-d668-4f29-b2a7-0c9229a0dc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:28.907993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0424e58a-d668-4f29-b2a7-0c9229a0dc6d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:28.910192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0424e58a-d668-4f29-b2a7-0c9229a0dc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:28.930470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1094/2221] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:28.935119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cb957-c0db-4230-b4b7-8ab9983d2e2b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:28.947623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cb957-c0db-4230-b4b7-8ab9983d2e2b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:28.958355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:28.959133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:28.959605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:28.959889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:28.960704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:28.961009 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:28.963050 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:28.971319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424e58a-d668-4f29-b2a7-0c9229a0dc6d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:28.972782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0424e58a-d668-4f29-b2a7-0c9229a0dc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:28.982840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1128/2222] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:28.987215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:28.994941 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:28.997703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424e58a-d668-4f29-b2a7-0c9229a0dc6d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.000591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1905b1-926d-48c7-a2be-e3fde429655c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.017891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.018625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe73cc-5ca1-43f6-8ba1-e04a65f53015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.018876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.019163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.019450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] PATH_INFO: `/v3/users/adc35814ced5463fa4b5e9284b14a0e4` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.019639 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe73cc-5ca1-43f6-8ba1-e04a65f53015 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.020033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe73cc-5ca1-43f6-8ba1-e04a65f53015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.021486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe73cc-5ca1-43f6-8ba1-e04a65f53015 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.022077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adc35814ced5463fa4b5e9284b14a0e4)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.022937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe73cc-5ca1-43f6-8ba1-e04a65f53015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.024039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.026091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe73cc-5ca1-43f6-8ba1-e04a65f53015 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.076942 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1129/2223] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:28 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:29.081061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe73cc-5ca1-43f6-8ba1-e04a65f53015 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:29.096271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adc35814ced5463fa4b5e9284b14a0e4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:29.110245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adc35814ced5463fa4b5e9284b14a0e4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:29.120885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adc35814ced5463fa4b5e9284b14a0e4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:29.121926 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adc35814ced5463fa4b5e9284b14a0e4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:29.122735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a9470742-e693-4add-bb96-9457c349d504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adc35814ced5463fa4b5e9284b14a0e4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:29.154872 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1095/2224] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:28 2022] DELETE /identity/v3/users/adc35814ced5463fa4b5e9284b14a0e4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:29.165209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d8aaf-31c4-4ce4-8977-05566652f1a3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.165542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d8aaf-31c4-4ce4-8977-05566652f1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.168579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d8aaf-31c4-4ce4-8977-05566652f1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.184907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe73cc-5ca1-43f6-8ba1-e04a65f53015 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.205085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-219d8aaf-31c4-4ce4-8977-05566652f1a3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:29.209946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-219d8aaf-31c4-4ce4-8977-05566652f1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:29.223494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:29.270386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.271966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.272418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.272905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.273807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.276313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.303617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1130/2225] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:29 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:29.304494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1096/2226] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:29.313192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:29.315151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:29.315741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d8aaf-31c4-4ce4-8977-05566652f1a3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:29.317271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-219d8aaf-31c4-4ce4-8977-05566652f1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:29.331885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa88af-20d4-4cdc-a4c0-8940472f7569 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.337359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449968e-0596-4028-bf2a-08a270f8d91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.337979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449968e-0596-4028-bf2a-08a270f8d91a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.338417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449968e-0596-4028-bf2a-08a270f8d91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.339089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449968e-0596-4028-bf2a-08a270f8d91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.339509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449968e-0596-4028-bf2a-08a270f8d91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.341460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449968e-0596-4028-bf2a-08a270f8d91a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.353813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1131/2227] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:29.357451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449968e-0596-4028-bf2a-08a270f8d91a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:29.394884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449968e-0596-4028-bf2a-08a270f8d91a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.401906 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d8aaf-31c4-4ce4-8977-05566652f1a3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.406304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19c8e7-3963-4015-98ee-c2156a955b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.408579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92972dfa-2735-4421-870d-b30187499709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.408883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19c8e7-3963-4015-98ee-c2156a955b70 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.409722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19c8e7-3963-4015-98ee-c2156a955b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.410057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19c8e7-3963-4015-98ee-c2156a955b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.410593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92972dfa-2735-4421-870d-b30187499709 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.412402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92972dfa-2735-4421-870d-b30187499709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.414049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92972dfa-2735-4421-870d-b30187499709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.415421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92972dfa-2735-4421-870d-b30187499709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.417112 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92972dfa-2735-4421-870d-b30187499709 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.427599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19c8e7-3963-4015-98ee-c2156a955b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.429347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19c8e7-3963-4015-98ee-c2156a955b70 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.440112 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1097/2228] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:29.445074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1132/2229] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:29.447330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19c8e7-3963-4015-98ee-c2156a955b70 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:29.464839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19c8e7-3963-4015-98ee-c2156a955b70 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.469379 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09220b-a6ee-45cd-a04c-d039b3e9d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.470050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09220b-a6ee-45cd-a04c-d039b3e9d09c admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.470470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09220b-a6ee-45cd-a04c-d039b3e9d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.470753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09220b-a6ee-45cd-a04c-d039b3e9d09c admin admin] PATH_INFO: `/v3/projects/dbeaed4f82ab47179151cc5668616823/users/787f0fe5effb45768770ffc693768a49/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.488303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09220b-a6ee-45cd-a04c-d039b3e9d09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbeaed4f82ab47179151cc5668616823, user_id=787f0fe5effb45768770ffc693768a49, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.492426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09220b-a6ee-45cd-a04c-d039b3e9d09c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.518276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1098/2230] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:29 2022] PUT /identity/v3/projects/dbeaed4f82ab47179151cc5668616823/users/787f0fe5effb45768770ffc693768a49/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:29.525293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2303f96-635c-4e51-9815-36368d57dfaf None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.526138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2303f96-635c-4e51-9815-36368d57dfaf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.526422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2303f96-635c-4e51-9815-36368d57dfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.578252 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c2303f96-635c-4e51-9815-36368d57dfaf None None] MFA Rules not processed for user `787f0fe5effb45768770ffc693768a49`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:29.580088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c2303f96-635c-4e51-9815-36368d57dfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:29.643428 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:29.681925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1133/2231] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:29.686480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09220b-a6ee-45cd-a04c-d039b3e9d09c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:29.778234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09220b-a6ee-45cd-a04c-d039b3e9d09c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.786028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820826c0-91a6-452d-8882-d490a6f6cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.787422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820826c0-91a6-452d-8882-d490a6f6cfad admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.787632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820826c0-91a6-452d-8882-d490a6f6cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.787938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820826c0-91a6-452d-8882-d490a6f6cfad admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.788612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820826c0-91a6-452d-8882-d490a6f6cfad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.790511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820826c0-91a6-452d-8882-d490a6f6cfad admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.804763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2303f96-635c-4e51-9815-36368d57dfaf None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:29.806412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c2303f96-635c-4e51-9815-36368d57dfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:29.828367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2303f96-635c-4e51-9815-36368d57dfaf None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.836261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1099/2232] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:29 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:29.840838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:29.841686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820826c0-91a6-452d-8882-d490a6f6cfad admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:29.842332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:29.842633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:29.842912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] PATH_INFO: `/v3/projects/44a71170fb764e35b01742799b866561` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:29.846908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a71170fb764e35b01742799b866561)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:29.849347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:29.902831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:29.973195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a71170fb764e35b01742799b866561', 'request_id': 'req-73ae0f5b-f055-4ccd-8779-63d27297856f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:29.982404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820826c0-91a6-452d-8882-d490a6f6cfad admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:29.986353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a71170fb764e35b01742799b866561', 'request_id': 'req-73ae0f5b-f055-4ccd-8779-63d27297856f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:29.997371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a71170fb764e35b01742799b866561', 'request_id': 'req-73ae0f5b-f055-4ccd-8779-63d27297856f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:30.020048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] The token cache is being invalidate because project 44a71170fb764e35b01742799b866561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:30.020342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:30.023093 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:30.024558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1134/2233] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:29 2022] DELETE /identity/v3/projects/44a71170fb764e35b01742799b866561 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:30.029866 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:30.031193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:30.155274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:30.156467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.156839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.157175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.158095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:30.160115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:30.207729 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1100/2234] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:29 2022] POST /identity/v3/users => generated 289 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:30.216119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:30.217753 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0f5b-f055-4ccd-8779-63d27297856f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:30.222471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:30.237083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:30.237987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.238222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.238596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] PATH_INFO: `/v3/users/1c5008993d6f43cdbb8ac0bb23fc383b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.242574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5008993d6f43cdbb8ac0bb23fc383b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:30.246989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:30.249001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7de45-d60c-4ac4-8741-88f7b2913fd1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:30.259374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6b08-331d-4309-a551-561a4c75d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:30.260102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6b08-331d-4309-a551-561a4c75d532 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.260963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6b08-331d-4309-a551-561a4c75d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.261520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6b08-331d-4309-a551-561a4c75d532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.262639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6b08-331d-4309-a551-561a4c75d532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:30.264755 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6b08-331d-4309-a551-561a4c75d532 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:30.284748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1101/2235] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:30.287200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6b08-331d-4309-a551-561a4c75d532 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:30.314088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6b08-331d-4309-a551-561a4c75d532 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:30.321403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28db37-d871-47ab-9fc7-428aa0f26fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:30.322293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28db37-d871-47ab-9fc7-428aa0f26fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.322677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28db37-d871-47ab-9fc7-428aa0f26fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.323091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28db37-d871-47ab-9fc7-428aa0f26fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.324116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28db37-d871-47ab-9fc7-428aa0f26fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:30.326140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28db37-d871-47ab-9fc7-428aa0f26fc4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:30.346298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1102/2236] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:30.347555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28db37-d871-47ab-9fc7-428aa0f26fc4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:30.359079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28db37-d871-47ab-9fc7-428aa0f26fc4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:30.366128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077599e-6e6c-42cb-8639-ff2db456ff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:30.367271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5008993d6f43cdbb8ac0bb23fc383b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:30.367634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077599e-6e6c-42cb-8639-ff2db456ff4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.368022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077599e-6e6c-42cb-8639-ff2db456ff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.368446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077599e-6e6c-42cb-8639-ff2db456ff4a admin admin] PATH_INFO: `/v3/projects/65a43c9033384f73b262335dbf9e1544/users/80cb2901337148b9928d1c3c5c72e207/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.380679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5008993d6f43cdbb8ac0bb23fc383b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:30.396907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5008993d6f43cdbb8ac0bb23fc383b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:30.399074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5008993d6f43cdbb8ac0bb23fc383b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:30.400227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5008993d6f43cdbb8ac0bb23fc383b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:30.407360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077599e-6e6c-42cb-8639-ff2db456ff4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a43c9033384f73b262335dbf9e1544, user_id=80cb2901337148b9928d1c3c5c72e207, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:30.411232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077599e-6e6c-42cb-8639-ff2db456ff4a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:30.423277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:30.429877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1103/2237] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:30 2022] PUT /identity/v3/projects/65a43c9033384f73b262335dbf9e1544/users/80cb2901337148b9928d1c3c5c72e207/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:30.440578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c99a14-a676-4142-a199-c350d88bb9f6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.441218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c99a14-a676-4142-a199-c350d88bb9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.441218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c99a14-a676-4142-a199-c350d88bb9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.448522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1135/2238] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:30 2022] DELETE /identity/v3/users/1c5008993d6f43cdbb8ac0bb23fc383b => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:30.517937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c8c99a14-a676-4142-a199-c350d88bb9f6 None None] MFA Rules not processed for user `80cb2901337148b9928d1c3c5c72e207`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:30.520134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c8c99a14-a676-4142-a199-c350d88bb9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:30.625035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1104/2239] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:30.629134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:30.631795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:30.726046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be0ced-13d0-4754-80c1-30bc4b5fb1fa admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:30.736319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:30.737348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.737861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.738350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.741605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:30.743687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:30.808636 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1136/2240] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:30 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:30.816246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c99a14-a676-4142-a199-c350d88bb9f6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:30.826535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:30.837676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:30.839721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:30.843989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c99a14-a676-4142-a199-c350d88bb9f6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:30.949323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a843d-7703-48ae-9c88-8b754946950b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:30.954836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:30.956160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.956656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.957144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] PATH_INFO: `/v3/projects/71bbe1bfa84b43b19b00e6cfd7f62d50` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.959494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71bbe1bfa84b43b19b00e6cfd7f62d50)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:30.961894 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170daa5-4d57-42ec-97a2-97b10073cd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:30.962487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:30.962909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170daa5-4d57-42ec-97a2-97b10073cd85 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:30.963194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170daa5-4d57-42ec-97a2-97b10073cd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:30.963468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170daa5-4d57-42ec-97a2-97b10073cd85 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:30.964279 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170daa5-4d57-42ec-97a2-97b10073cd85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:30.968223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170daa5-4d57-42ec-97a2-97b10073cd85 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:31.010959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1105/2241] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:30 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:31.015960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170daa5-4d57-42ec-97a2-97b10073cd85 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:31.039266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170daa5-4d57-42ec-97a2-97b10073cd85 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:31.063286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f8045-7d12-4a13-924a-597183dbcc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:31.064078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f8045-7d12-4a13-924a-597183dbcc4c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:31.064713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f8045-7d12-4a13-924a-597183dbcc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:31.068826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f8045-7d12-4a13-924a-597183dbcc4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:31.069638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f8045-7d12-4a13-924a-597183dbcc4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:31.071419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f8045-7d12-4a13-924a-597183dbcc4c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:31.082882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71bbe1bfa84b43b19b00e6cfd7f62d50', 'request_id': 'req-11385971-4b89-4b15-9629-0544c018893c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.092217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71bbe1bfa84b43b19b00e6cfd7f62d50', 'request_id': 'req-11385971-4b89-4b15-9629-0544c018893c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.093554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1106/2242] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:31.101947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f8045-7d12-4a13-924a-597183dbcc4c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:31.106170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71bbe1bfa84b43b19b00e6cfd7f62d50', 'request_id': 'req-11385971-4b89-4b15-9629-0544c018893c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.122733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f8045-7d12-4a13-924a-597183dbcc4c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:31.125666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] The token cache is being invalidate because project 71bbe1bfa84b43b19b00e6cfd7f62d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:31.126071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.126932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.128812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1137/2243] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:30 2022] DELETE /identity/v3/projects/71bbe1bfa84b43b19b00e6cfd7f62d50 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:31.134704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:31.136541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:31.144664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:31.257308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:31.258671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:31.259347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:31.259881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] PATH_INFO: `/v3/projects/e4d4855d8fc24ab2b294a90b4d355b3e/users/2f1240973c0c410392531b0114740764/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:31.297020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d4855d8fc24ab2b294a90b4d355b3e, user_id=2f1240973c0c410392531b0114740764, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:31.305201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:31.319944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11385971-4b89-4b15-9629-0544c018893c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:31.333763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1107/2244] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:31 2022] PUT /identity/v3/projects/e4d4855d8fc24ab2b294a90b4d355b3e/users/2f1240973c0c410392531b0114740764/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:31.337566 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:31.340541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:31.493749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:31.494367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:31.494654 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:31.494978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] PATH_INFO: `/v3/users/fd8eb201bd2e49bb9a0192501682ba2d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:31.497968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8eb201bd2e49bb9a0192501682ba2d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:31.499946 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:31.501245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f3372-a3a6-4e09-ab93-2789588951b8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:31.511412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eed92b-1da8-4739-9092-81983864b64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:31.513662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed92b-1da8-4739-9092-81983864b64b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:31.514511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed92b-1da8-4739-9092-81983864b64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:31.514942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eed92b-1da8-4739-9092-81983864b64b admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:31.515943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eed92b-1da8-4739-9092-81983864b64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:31.517838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eed92b-1da8-4739-9092-81983864b64b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:31.527372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:31.534467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1108/2245] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:31.569452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eed92b-1da8-4739-9092-81983864b64b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:31.596735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eed92b-1da8-4739-9092-81983864b64b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:31.602633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f646f98-d9a0-4f04-8763-940dc4ca78d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:31.603648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f646f98-d9a0-4f04-8763-940dc4ca78d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:31.603959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f646f98-d9a0-4f04-8763-940dc4ca78d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:31.604227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f646f98-d9a0-4f04-8763-940dc4ca78d4 admin admin] PATH_INFO: `/v3/projects/e4d4855d8fc24ab2b294a90b4d355b3e/users/2f1240973c0c410392531b0114740764/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:31.606552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f646f98-d9a0-4f04-8763-940dc4ca78d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d4855d8fc24ab2b294a90b4d355b3e, user_id=2f1240973c0c410392531b0114740764, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:31.609918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f646f98-d9a0-4f04-8763-940dc4ca78d4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:31.635997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1109/2246] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:31 2022] PUT /identity/v3/projects/e4d4855d8fc24ab2b294a90b4d355b3e/users/2f1240973c0c410392531b0114740764/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:31.639394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8eb201bd2e49bb9a0192501682ba2d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.648122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b226d9-3f27-4c8d-8c5f-5f5c6b3c5db9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:31.648526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b226d9-3f27-4c8d-8c5f-5f5c6b3c5db9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:31.648808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b226d9-3f27-4c8d-8c5f-5f5c6b3c5db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:31.650914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8eb201bd2e49bb9a0192501682ba2d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.663221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8eb201bd2e49bb9a0192501682ba2d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.664484 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8eb201bd2e49bb9a0192501682ba2d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.667137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8eb201bd2e49bb9a0192501682ba2d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:31.692368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-22b226d9-3f27-4c8d-8c5f-5f5c6b3c5db9 None None] MFA Rules not processed for user `2f1240973c0c410392531b0114740764`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:31.694309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-22b226d9-3f27-4c8d-8c5f-5f5c6b3c5db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:31.706011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1138/2247] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:31 2022] DELETE /identity/v3/users/fd8eb201bd2e49bb9a0192501682ba2d => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:31.806924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1110/2248] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:31.819814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:31.821572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:31.899955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:31.939917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97939b6f-8b70-4d22-881e-47bd2b63f69e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:31.955163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:31.955982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:31.956323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:31.957300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:31.960248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:32.051599 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:32.055085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:32.078176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1139/2249] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:32.107830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb1ba-f51a-4066-943d-cc37f31b6c40 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:32.108127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb1ba-f51a-4066-943d-cc37f31b6c40 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:32.108390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb1ba-f51a-4066-943d-cc37f31b6c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:32.157413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3cdbb1ba-f51a-4066-943d-cc37f31b6c40 None None] MFA Rules not processed for user `05147fb1762748ee9866ccc5bff4d514`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:32.158638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3cdbb1ba-f51a-4066-943d-cc37f31b6c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:32.349079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1111/2250] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:32.363111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:32.365087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:32.609447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b17de-4247-47ca-9faa-1464dac63b91 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:32.632665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ef94f-032f-42d0-8790-2afcd4dbe7b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:32.639553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbb1ba-f51a-4066-943d-cc37f31b6c40 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:32.640059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef94f-032f-42d0-8790-2afcd4dbe7b8 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:32.641578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef94f-032f-42d0-8790-2afcd4dbe7b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:32.641878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef94f-032f-42d0-8790-2afcd4dbe7b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:32.642180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3cdbb1ba-f51a-4066-943d-cc37f31b6c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:32.702282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ef94f-032f-42d0-8790-2afcd4dbe7b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:32.714596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ef94f-032f-42d0-8790-2afcd4dbe7b8 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:32.741049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1140/2251] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:32.940976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbb1ba-f51a-4066-943d-cc37f31b6c40 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:32.953890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:32.961145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:32.961145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:32.961145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] PATH_INFO: `/v3/projects/d6b23c29768440258d753c3c94892825` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:32.963138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b23c29768440258d753c3c94892825)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:32.965351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:32.974177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:33.107266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b23c29768440258d753c3c94892825', 'request_id': 'req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:33.114082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b23c29768440258d753c3c94892825', 'request_id': 'req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:33.129344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b23c29768440258d753c3c94892825', 'request_id': 'req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:33.146594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] The token cache is being invalidate because project d6b23c29768440258d753c3c94892825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:33.147461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:33.148726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-43ed0ed9-1fb4-40ab-8066-46c55c887fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:33.151542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1112/2252] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:32 2022] DELETE /identity/v3/projects/d6b23c29768440258d753c3c94892825 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:33.158436 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28adb13-14fc-4011-aec1-9880edf186da None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.158914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28adb13-14fc-4011-aec1-9880edf186da None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.159026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28adb13-14fc-4011-aec1-9880edf186da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.203412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d28adb13-14fc-4011-aec1-9880edf186da None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:33.204882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d28adb13-14fc-4011-aec1-9880edf186da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:33.291344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1141/2253] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:33.293970 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd14d9-b00b-4146-a125-5daa7354cc4e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.294321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd14d9-b00b-4146-a125-5daa7354cc4e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.294768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd14d9-b00b-4146-a125-5daa7354cc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.378386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-79dd14d9-b00b-4146-a125-5daa7354cc4e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:33.379933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-79dd14d9-b00b-4146-a125-5daa7354cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:33.403158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1113/2254] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:33.409702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28adb13-14fc-4011-aec1-9880edf186da None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:33.421179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28adb13-14fc-4011-aec1-9880edf186da None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:33.430221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:33.430849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.431128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.431400 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.432069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:33.432346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:33.433986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:33.442987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:33.454184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1142/2255] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:33.454696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd14d9-b00b-4146-a125-5daa7354cc4e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:33.471133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd14d9-b00b-4146-a125-5daa7354cc4e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:33.477329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e5c7c-9623-49fe-9122-4e9e650c946f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:33.477954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e5c7c-9623-49fe-9122-4e9e650c946f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.478231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e5c7c-9623-49fe-9122-4e9e650c946f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.479024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e5c7c-9623-49fe-9122-4e9e650c946f admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.479136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e5c7c-9623-49fe-9122-4e9e650c946f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:33.481091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e5c7c-9623-49fe-9122-4e9e650c946f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:33.524823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1114/2256] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:33 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:33.528109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:33.549950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35776e78-ed8a-4076-b143-57838e8193a6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:33.636197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4cc54-4aff-4d04-9337-3d7077bdb1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:33.636849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4cc54-4aff-4d04-9337-3d7077bdb1ac admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.637125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4cc54-4aff-4d04-9337-3d7077bdb1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.637396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4cc54-4aff-4d04-9337-3d7077bdb1ac admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.638025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4cc54-4aff-4d04-9337-3d7077bdb1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:33.639958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4cc54-4aff-4d04-9337-3d7077bdb1ac admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:33.662912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1143/2257] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:33 2022] POST /identity/v3/users => generated 291 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:33.667140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e5c7c-9623-49fe-9122-4e9e650c946f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:33.679170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e5c7c-9623-49fe-9122-4e9e650c946f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:33.684374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee2ec9-fdbd-4460-bdc9-e43cfd191507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:33.685039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee2ec9-fdbd-4460-bdc9-e43cfd191507 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.685321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee2ec9-fdbd-4460-bdc9-e43cfd191507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.685593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee2ec9-fdbd-4460-bdc9-e43cfd191507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.686252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee2ec9-fdbd-4460-bdc9-e43cfd191507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:33.690454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee2ec9-fdbd-4460-bdc9-e43cfd191507 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:33.715065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1115/2258] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:33.719532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4cc54-4aff-4d04-9337-3d7077bdb1ac admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:33.734494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4cc54-4aff-4d04-9337-3d7077bdb1ac admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:33.739352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda24d2-0861-4f74-a828-e9026553bd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:33.740128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda24d2-0861-4f74-a828-e9026553bd11 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.740409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda24d2-0861-4f74-a828-e9026553bd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.740712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda24d2-0861-4f74-a828-e9026553bd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.741355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda24d2-0861-4f74-a828-e9026553bd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:33.742963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda24d2-0861-4f74-a828-e9026553bd11 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:33.756350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1144/2259] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:33.761424 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee2ec9-fdbd-4460-bdc9-e43cfd191507 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:33.772503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee2ec9-fdbd-4460-bdc9-e43cfd191507 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:33.777714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c023f-a89e-4971-a5bb-6247d8a0bbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:33.778601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c023f-a89e-4971-a5bb-6247d8a0bbe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.779055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c023f-a89e-4971-a5bb-6247d8a0bbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.779475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c023f-a89e-4971-a5bb-6247d8a0bbe6 admin admin] PATH_INFO: `/v3/projects/51b22037c7db47faaa7f0931e36a56d7/users/9dc0e6b1bf56483cb425ac826911e85b/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.797777 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c023f-a89e-4971-a5bb-6247d8a0bbe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b22037c7db47faaa7f0931e36a56d7, user_id=9dc0e6b1bf56483cb425ac826911e85b, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:33.801477 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c023f-a89e-4971-a5bb-6247d8a0bbe6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:33.830625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9227c2f-a25a-4d13-b368-640d72397be2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:33.831104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9227c2f-a25a-4d13-b368-640d72397be2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:33.831867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9227c2f-a25a-4d13-b368-640d72397be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:33.836561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1116/2260] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:33 2022] PUT /identity/v3/projects/51b22037c7db47faaa7f0931e36a56d7/users/9dc0e6b1bf56483cb425ac826911e85b/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:33.898986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d9227c2f-a25a-4d13-b368-640d72397be2 None None] MFA Rules not processed for user `9dc0e6b1bf56483cb425ac826911e85b`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:33.901960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d9227c2f-a25a-4d13-b368-640d72397be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:34.153270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1145/2261] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:34.165224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c023f-a89e-4971-a5bb-6247d8a0bbe6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:34.246640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9227c2f-a25a-4d13-b368-640d72397be2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:34.248193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d9227c2f-a25a-4d13-b368-640d72397be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:34.348475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9227c2f-a25a-4d13-b368-640d72397be2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:34.353525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:34.354808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:34.355100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:34.355378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:34.357744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:34.372154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c023f-a89e-4971-a5bb-6247d8a0bbe6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:34.386447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33268260-0f30-4cdc-8958-7912b775c1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:34.386447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268260-0f30-4cdc-8958-7912b775c1e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:34.386447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268260-0f30-4cdc-8958-7912b775c1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:34.386447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33268260-0f30-4cdc-8958-7912b775c1e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:34.386931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268260-0f30-4cdc-8958-7912b775c1e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:34.388045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33268260-0f30-4cdc-8958-7912b775c1e3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:34.405456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:34.452827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:34.454613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:34.455834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1117/2262] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:34 2022] POST /identity/v3/projects => generated 381 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:34.457707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0813af4-dd8a-4055-bb9e-86980c798291 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:34.458209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0813af4-dd8a-4055-bb9e-86980c798291 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:34.460822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0813af4-dd8a-4055-bb9e-86980c798291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:34.468758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1146/2263] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:34.469614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:34.482827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07899faf-6465-41b9-b92a-9965254e2eee service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:34.502374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c0813af4-dd8a-4055-bb9e-86980c798291 None None] MFA Rules not processed for user `eef1ea8d331444e892d0b30c1feef904`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:34.504933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c0813af4-dd8a-4055-bb9e-86980c798291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:34.525811 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:34.582676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78a219-7d9c-4bab-badf-455d63552e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:34.583460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a219-7d9c-4bab-badf-455d63552e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:34.583738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a219-7d9c-4bab-badf-455d63552e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:34.584008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a219-7d9c-4bab-badf-455d63552e60 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:34.584821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78a219-7d9c-4bab-badf-455d63552e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:34.587624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78a219-7d9c-4bab-badf-455d63552e60 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:34.635226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1147/2264] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:34 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:34.639199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78a219-7d9c-4bab-badf-455d63552e60 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:34.664844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78a219-7d9c-4bab-badf-455d63552e60 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:34.675251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197f00f-0d79-42a5-9dc2-9da4b0f1b405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:34.675884 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f00f-0d79-42a5-9dc2-9da4b0f1b405 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:34.676155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f00f-0d79-42a5-9dc2-9da4b0f1b405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:34.676427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f00f-0d79-42a5-9dc2-9da4b0f1b405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:34.677653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197f00f-0d79-42a5-9dc2-9da4b0f1b405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:34.682367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197f00f-0d79-42a5-9dc2-9da4b0f1b405 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:34.696764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1118/2265] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:34.717034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1148/2266] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:34.718474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0813af4-dd8a-4055-bb9e-86980c798291 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:34.750790 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0813af4-dd8a-4055-bb9e-86980c798291 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:34.780605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaf01-2a25-4a52-ab81-dde15897a7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:34.785078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaf01-2a25-4a52-ab81-dde15897a7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:34.787641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaf01-2a25-4a52-ab81-dde15897a7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:34.787946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eaf01-2a25-4a52-ab81-dde15897a7fb admin admin] PATH_INFO: `/v3/projects/9a1955fa4c7b468dbaba74074ff345c9/users/d0aa51ec323b4fd5b0649962524ba73a/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:34.842697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eaf01-2a25-4a52-ab81-dde15897a7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1955fa4c7b468dbaba74074ff345c9, user_id=d0aa51ec323b4fd5b0649962524ba73a, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:34.853396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eaf01-2a25-4a52-ab81-dde15897a7fb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:34.882071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1119/2267] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:34 2022] PUT /identity/v3/projects/9a1955fa4c7b468dbaba74074ff345c9/users/d0aa51ec323b4fd5b0649962524ba73a/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:34.886634 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197f00f-0d79-42a5-9dc2-9da4b0f1b405 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:34.904032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197f00f-0d79-42a5-9dc2-9da4b0f1b405 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:35.040485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be64bdd-a2f6-4e4b-b1ba-cf4ac8f8c82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:35.041528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64bdd-a2f6-4e4b-b1ba-cf4ac8f8c82d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:35.042020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64bdd-a2f6-4e4b-b1ba-cf4ac8f8c82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:35.042442 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64bdd-a2f6-4e4b-b1ba-cf4ac8f8c82d admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:35.043248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be64bdd-a2f6-4e4b-b1ba-cf4ac8f8c82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:35.046430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be64bdd-a2f6-4e4b-b1ba-cf4ac8f8c82d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:35.061111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1149/2268] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:35.065167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaf01-2a25-4a52-ab81-dde15897a7fb admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:35.084240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eaf01-2a25-4a52-ab81-dde15897a7fb admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:35.092634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:35.093789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:35.094081 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:35.094663 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] PATH_INFO: `/v3/projects/9a1955fa4c7b468dbaba74074ff345c9/users/d0aa51ec323b4fd5b0649962524ba73a/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:35.097061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1955fa4c7b468dbaba74074ff345c9, user_id=d0aa51ec323b4fd5b0649962524ba73a, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:35.101918 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:35.134173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1120/2269] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:35 2022] PUT /identity/v3/projects/9a1955fa4c7b468dbaba74074ff345c9/users/d0aa51ec323b4fd5b0649962524ba73a/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:35.137071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0b3e8-24c3-44e6-8a07-02b77183834f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:35.137838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0b3e8-24c3-44e6-8a07-02b77183834f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:35.138354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0b3e8-24c3-44e6-8a07-02b77183834f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:35.186526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-70e0b3e8-24c3-44e6-8a07-02b77183834f None None] MFA Rules not processed for user `d0aa51ec323b4fd5b0649962524ba73a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:35.189324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-70e0b3e8-24c3-44e6-8a07-02b77183834f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:35.271685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:35.397839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1150/2270] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:35.438100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:35.439841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:35.557199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0b3e8-24c3-44e6-8a07-02b77183834f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:35.587467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd5103-b6bd-4cd6-ad70-4d512bf7dada admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:35.593375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519407d-1fbb-4c8d-a697-aeb4b76ccc4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:35.594152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519407d-1fbb-4c8d-a697-aeb4b76ccc4e service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:35.594433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519407d-1fbb-4c8d-a697-aeb4b76ccc4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:35.594703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519407d-1fbb-4c8d-a697-aeb4b76ccc4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:35.608103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519407d-1fbb-4c8d-a697-aeb4b76ccc4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:35.610394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519407d-1fbb-4c8d-a697-aeb4b76ccc4e service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:35.622585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1121/2271] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:35.685360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0b3e8-24c3-44e6-8a07-02b77183834f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:35.688240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f30254-28b1-4d4a-a616-3828b882c0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:35.689751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30254-28b1-4d4a-a616-3828b882c0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:35.690172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30254-28b1-4d4a-a616-3828b882c0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:35.690640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30254-28b1-4d4a-a616-3828b882c0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:35.695018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f2f30254-28b1-4d4a-a616-3828b882c0f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:35.780269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519407d-1fbb-4c8d-a697-aeb4b76ccc4e service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:35.792514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519407d-1fbb-4c8d-a697-aeb4b76ccc4e service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:35.797205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:35.797852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:35.798306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:35.798590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:35.894848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f30254-28b1-4d4a-a616-3828b882c0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:35.897159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f30254-28b1-4d4a-a616-3828b882c0f6 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:35.921531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:35.923836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:35.928141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1151/2272] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:35.939387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1122/2273] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:36.181728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f77da-f268-4db2-ae06-d4b62f6f49ea None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:36.182012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f77da-f268-4db2-ae06-d4b62f6f49ea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:36.182288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f77da-f268-4db2-ae06-d4b62f6f49ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:36.221798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-111f77da-f268-4db2-ae06-d4b62f6f49ea None None] MFA Rules not processed for user `05147fb1762748ee9866ccc5bff4d514`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:36.225935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-111f77da-f268-4db2-ae06-d4b62f6f49ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:36.336220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1152/2274] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:37.740023 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:37.742593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:37.820646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:37.946358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e18940-aea6-4f68-9c9f-4cb386317c3f service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:37.953178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:37.954249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:37.954727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:37.955261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] PATH_INFO: `/v3/users/2ae084ecbe32411fa01c77781ded5723` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:37.957228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ae084ecbe32411fa01c77781ded5723)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:37.960795 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:38.090972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ae084ecbe32411fa01c77781ded5723'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:38.121635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ae084ecbe32411fa01c77781ded5723'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:38.155459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ae084ecbe32411fa01c77781ded5723'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:38.156585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ae084ecbe32411fa01c77781ded5723'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:38.157208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ae084ecbe32411fa01c77781ded5723'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:38.188179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1123/2275] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:37 2022] DELETE /identity/v3/users/2ae084ecbe32411fa01c77781ded5723 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:38.193374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dbcbe-89af-4cf0-9364-401abff98663 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:38.194037 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dbcbe-89af-4cf0-9364-401abff98663 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:38.194037 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dbcbe-89af-4cf0-9364-401abff98663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:38.230673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-767dbcbe-89af-4cf0-9364-401abff98663 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:38.232526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-767dbcbe-89af-4cf0-9364-401abff98663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:38.258917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:38.401217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1153/2276] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:38.414677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:38.416684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:38.553039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01b5c0-aa1c-4f28-89a3-ddfc5b183ce8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:38.570868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:38.571960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:38.572239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:38.572546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:38.586616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:38.589849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:38.604743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1124/2277] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:39.030963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dbcbe-89af-4cf0-9364-401abff98663 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.033362 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-767dbcbe-89af-4cf0-9364-401abff98663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:39.062469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dbcbe-89af-4cf0-9364-401abff98663 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.073878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.074708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.074991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.075317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] PATH_INFO: `/v3/projects/d57e79a85032404097c1fb618b22b2cf` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.077012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57e79a85032404097c1fb618b22b2cf)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.080113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.173079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57e79a85032404097c1fb618b22b2cf', 'request_id': 'req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.196176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57e79a85032404097c1fb618b22b2cf', 'request_id': 'req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.216964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57e79a85032404097c1fb618b22b2cf', 'request_id': 'req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.238461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.242772 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:39.255606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] The token cache is being invalidate because project d57e79a85032404097c1fb618b22b2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:39.259120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.263822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8a32a66f-0725-45c5-9a4f-5d88f85cd72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.272036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1154/2278] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:39 2022] DELETE /identity/v3/projects/d57e79a85032404097c1fb618b22b2cf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:39.272618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28962d-a462-4861-aac5-7a64e395ccf8 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.277336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce110c2-0cc4-49f7-be96-172364667354 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.277871 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce110c2-0cc4-49f7-be96-172364667354 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.278148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce110c2-0cc4-49f7-be96-172364667354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.340978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3ce110c2-0cc4-49f7-be96-172364667354 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:39.342218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3ce110c2-0cc4-49f7-be96-172364667354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:39.392464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:39.416292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.417482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.417765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.418351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] PATH_INFO: `/v3/users/9dc0e6b1bf56483cb425ac826911e85b` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.419727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc0e6b1bf56483cb425ac826911e85b)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.422766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.438039 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1155/2279] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:39.448950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199d0c8-b34e-4547-b96d-ba3092a4343d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.449521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199d0c8-b34e-4547-b96d-ba3092a4343d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.449944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199d0c8-b34e-4547-b96d-ba3092a4343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.472136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7199d0c8-b34e-4547-b96d-ba3092a4343d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:39.473559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7199d0c8-b34e-4547-b96d-ba3092a4343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:39.485302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1156/2280] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:39.490538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199d0c8-b34e-4547-b96d-ba3092a4343d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.501647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199d0c8-b34e-4547-b96d-ba3092a4343d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.502960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc0e6b1bf56483cb425ac826911e85b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.507490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.508190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.508578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.508850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.509506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:39.509805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.511489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.514211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc0e6b1bf56483cb425ac826911e85b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.525252 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1157/2281] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:39.534118 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.535627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc0e6b1bf56483cb425ac826911e85b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.537183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc0e6b1bf56483cb425ac826911e85b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.538610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3c9d65e1-f640-406d-85d7-a86bd75b4d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc0e6b1bf56483cb425ac826911e85b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:39.558056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeae0dd-1761-44b9-919c-7988137ce311 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.568361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.568361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.568862 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.569203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.569847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.572028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.573752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1125/2282] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:39 2022] DELETE /identity/v3/users/9dc0e6b1bf56483cb425ac826911e85b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:39.578007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c336b3-a0ce-439b-80bd-8575ccd25770 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.578296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c336b3-a0ce-439b-80bd-8575ccd25770 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.578572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c336b3-a0ce-439b-80bd-8575ccd25770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.610489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:39.617999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f9c336b3-a0ce-439b-80bd-8575ccd25770 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:39.620550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f9c336b3-a0ce-439b-80bd-8575ccd25770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:39.626336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1158/2283] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:39 2022] POST /identity/v3/projects => generated 363 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:39.630978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.633157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:39.706079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1126/2284] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:39.716805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c336b3-a0ce-439b-80bd-8575ccd25770 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.716992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bf3e9-dea5-4839-8f5e-35c273cd8901 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.718196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f9c336b3-a0ce-439b-80bd-8575ccd25770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:39.722920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f71645-8bca-4714-8c88-a5968143c771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.724809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f71645-8bca-4714-8c88-a5968143c771 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.725331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f71645-8bca-4714-8c88-a5968143c771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.725608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f71645-8bca-4714-8c88-a5968143c771 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.730311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f71645-8bca-4714-8c88-a5968143c771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.732323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f71645-8bca-4714-8c88-a5968143c771 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.773438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1159/2285] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:39 2022] POST /identity/v3/users => generated 282 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:39.773988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f71645-8bca-4714-8c88-a5968143c771 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.791344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f71645-8bca-4714-8c88-a5968143c771 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.797260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b6c78-5937-4021-912b-4d5d262161f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.797807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b6c78-5937-4021-912b-4d5d262161f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.798430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b6c78-5937-4021-912b-4d5d262161f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.798707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b6c78-5937-4021-912b-4d5d262161f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.799496 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c336b3-a0ce-439b-80bd-8575ccd25770 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.799671 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b6c78-5937-4021-912b-4d5d262161f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.802746 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b6c78-5937-4021-912b-4d5d262161f9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.821408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1160/2286] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:39.825425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:39.826900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b6c78-5937-4021-912b-4d5d262161f9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.844772 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b6c78-5937-4021-912b-4d5d262161f9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.851062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70feba5-3687-48d4-a578-8ac41d9ff96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.851892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70feba5-3687-48d4-a578-8ac41d9ff96f admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.852171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70feba5-3687-48d4-a578-8ac41d9ff96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.852547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70feba5-3687-48d4-a578-8ac41d9ff96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.853265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70feba5-3687-48d4-a578-8ac41d9ff96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.864763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70feba5-3687-48d4-a578-8ac41d9ff96f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.872578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39169a6-c251-4b10-9780-da69be24127b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.873389 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39169a6-c251-4b10-9780-da69be24127b service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.873731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39169a6-c251-4b10-9780-da69be24127b service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.874090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39169a6-c251-4b10-9780-da69be24127b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.875525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1161/2287] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:39.880553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70feba5-3687-48d4-a578-8ac41d9ff96f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:39.890743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39169a6-c251-4b10-9780-da69be24127b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.892347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39169a6-c251-4b10-9780-da69be24127b service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.894174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70feba5-3687-48d4-a578-8ac41d9ff96f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:39.905361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:39.906256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.906627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.907020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1127/2288] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:39.907374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] PATH_INFO: `/v3/projects/f524447f3a87499f804b0cd2acf58cc9/users/deca408116c841cdbfbe0a83ca240c5e/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:39.956030 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f524447f3a87499f804b0cd2acf58cc9, user_id=deca408116c841cdbfbe0a83ca240c5e, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:39.961088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:39.988564 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1162/2289] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:39 2022] PUT /identity/v3/projects/f524447f3a87499f804b0cd2acf58cc9/users/deca408116c841cdbfbe0a83ca240c5e/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:39.998568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d65ba-dcde-47b0-af89-5b321bbd2af1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:39.998997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d65ba-dcde-47b0-af89-5b321bbd2af1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:39.999333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d65ba-dcde-47b0-af89-5b321bbd2af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:40.031530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ec3d65ba-dcde-47b0-af89-5b321bbd2af1 None None] MFA Rules not processed for user `deca408116c841cdbfbe0a83ca240c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:40.033180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ec3d65ba-dcde-47b0-af89-5b321bbd2af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:40.120768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1128/2290] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:40.135874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:40.138284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:40.244378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a15c6-fa99-4149-afe4-2667074c8ce1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:40.250779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:40.251669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:40.252102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:40.253882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:40.266398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:40.268876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:40.295645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1163/2291] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:40.323674 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d65ba-dcde-47b0-af89-5b321bbd2af1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:40.325712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ec3d65ba-dcde-47b0-af89-5b321bbd2af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:40.493640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d65ba-dcde-47b0-af89-5b321bbd2af1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:40.502083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:40.503530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:40.503727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:40.504546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:40.504965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:40.505393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] PATH_INFO: `/v3/projects/51b22037c7db47faaa7f0931e36a56d7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:40.507120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b22037c7db47faaa7f0931e36a56d7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:40.509801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:40.537298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df591c10-6ca3-4406-8569-b1792f1d1ddc service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:40.543532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:40.544310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:40.544707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:40.545088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] PATH_INFO: `/v3/users/787f0fe5effb45768770ffc693768a49` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:40.546660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=787f0fe5effb45768770ffc693768a49)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:40.549190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:40.593069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b22037c7db47faaa7f0931e36a56d7', 'request_id': 'req-99c4fe0d-90bf-42b4-9091-2a22041d906e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.600939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:40.603623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b22037c7db47faaa7f0931e36a56d7', 'request_id': 'req-99c4fe0d-90bf-42b4-9091-2a22041d906e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.616651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b22037c7db47faaa7f0931e36a56d7', 'request_id': 'req-99c4fe0d-90bf-42b4-9091-2a22041d906e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.634547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] The token cache is being invalidate because project 51b22037c7db47faaa7f0931e36a56d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:40.634935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.635773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.636429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:40.643973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1129/2292] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:40 2022] DELETE /identity/v3/projects/51b22037c7db47faaa7f0931e36a56d7 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:40.645332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:40.648491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:40.655493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787f0fe5effb45768770ffc693768a49'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.673524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787f0fe5effb45768770ffc693768a49'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.688261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787f0fe5effb45768770ffc693768a49'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.688955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787f0fe5effb45768770ffc693768a49'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.690846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-59f35563-a6e2-4227-8c0d-a2d660b3f61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787f0fe5effb45768770ffc693768a49'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:40.733033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1164/2293] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:40 2022] DELETE /identity/v3/users/787f0fe5effb45768770ffc693768a49 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:40.735876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ba507-f97d-4d02-a6f9-563f0d25b0fb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:40.736152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ba507-f97d-4d02-a6f9-563f0d25b0fb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:40.736433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ba507-f97d-4d02-a6f9-563f0d25b0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:40.755299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4fe0d-90bf-42b4-9091-2a22041d906e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:40.787880 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8e3ba507-f97d-4d02-a6f9-563f0d25b0fb None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:40.789922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8e3ba507-f97d-4d02-a6f9-563f0d25b0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:40.897125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:40.899807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:40.900095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:40.900373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] PATH_INFO: `/v3/users/d0aa51ec323b4fd5b0649962524ba73a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:40.902641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0aa51ec323b4fd5b0649962524ba73a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:40.904628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:40.925778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1165/2294] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:40.936270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ba507-f97d-4d02-a6f9-563f0d25b0fb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:40.937811 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8e3ba507-f97d-4d02-a6f9-563f0d25b0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:41.014421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0aa51ec323b4fd5b0649962524ba73a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.030053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0aa51ec323b4fd5b0649962524ba73a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.041785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0aa51ec323b4fd5b0649962524ba73a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.043275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0aa51ec323b4fd5b0649962524ba73a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.043956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0aa51ec323b4fd5b0649962524ba73a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.045305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ba507-f97d-4d02-a6f9-563f0d25b0fb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:41.050968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:41.051603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:41.053188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:41.053494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:41.057809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:41.077936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1130/2295] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:40 2022] DELETE /identity/v3/users/d0aa51ec323b4fd5b0649962524ba73a => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:41.082494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:41.089993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:41.103754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1166/2296] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:15:41.632965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:41.634474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:41.646308 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:41.715494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd9a3-76a7-4095-ade6-d24417e25355 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:41.720888 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:41.721532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:41.721815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:41.722449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] PATH_INFO: `/v3/projects/dbeaed4f82ab47179151cc5668616823` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:41.723258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbeaed4f82ab47179151cc5668616823)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:41.725470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:41.762810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:41.767048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:41.783553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbeaed4f82ab47179151cc5668616823', 'request_id': 'req-aacf320e-d628-4dc8-8955-c54307bc67af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.798124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbeaed4f82ab47179151cc5668616823', 'request_id': 'req-aacf320e-d628-4dc8-8955-c54307bc67af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.805992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1a5c6-5db5-45be-ae16-a0a5debd9edc service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:41.821899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbeaed4f82ab47179151cc5668616823', 'request_id': 'req-aacf320e-d628-4dc8-8955-c54307bc67af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.838980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] The token cache is being invalidate because project dbeaed4f82ab47179151cc5668616823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:41.839279 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.840584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:41.844483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1131/2297] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:41 2022] DELETE /identity/v3/projects/dbeaed4f82ab47179151cc5668616823 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:41.846518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:41.849088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:41.908702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:41.909324 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:41.909611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:41.909874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] PATH_INFO: `/v3/projects/9a1955fa4c7b468dbaba74074ff345c9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:41.915016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1955fa4c7b468dbaba74074ff345c9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:41.921197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:41.923474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:41.966292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf320e-d628-4dc8-8955-c54307bc67af admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:41.973270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:41.975139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:41.975511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:41.976082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] PATH_INFO: `/v3/users/80cb2901337148b9928d1c3c5c72e207` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:41.977890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80cb2901337148b9928d1c3c5c72e207)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:41.979864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:42.013524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1955fa4c7b468dbaba74074ff345c9', 'request_id': 'req-21abd001-2e9d-4fd4-92d7-57398b321c7b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.029124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1955fa4c7b468dbaba74074ff345c9', 'request_id': 'req-21abd001-2e9d-4fd4-92d7-57398b321c7b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.047258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1955fa4c7b468dbaba74074ff345c9', 'request_id': 'req-21abd001-2e9d-4fd4-92d7-57398b321c7b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.063164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:42.077233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80cb2901337148b9928d1c3c5c72e207'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.084472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] The token cache is being invalidate because project 9a1955fa4c7b468dbaba74074ff345c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:42.084472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.085403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-21abd001-2e9d-4fd4-92d7-57398b321c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.089522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80cb2901337148b9928d1c3c5c72e207'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.095155 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1167/2298] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:41 2022] DELETE /identity/v3/projects/9a1955fa4c7b468dbaba74074ff345c9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:42.099467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f12d59-84e5-4a82-9677-b223758ef48e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:42.099935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f12d59-84e5-4a82-9677-b223758ef48e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:42.100196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f12d59-84e5-4a82-9677-b223758ef48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:42.119486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80cb2901337148b9928d1c3c5c72e207'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.120814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80cb2901337148b9928d1c3c5c72e207'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.123130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-13ba7c6e-8bab-429b-8d33-bb5210c40773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80cb2901337148b9928d1c3c5c72e207'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.142469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-98f12d59-84e5-4a82-9677-b223758ef48e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:42.144765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-98f12d59-84e5-4a82-9677-b223758ef48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:42.149048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1132/2299] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:41 2022] DELETE /identity/v3/users/80cb2901337148b9928d1c3c5c72e207 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:42.262810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1168/2300] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:42.270231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dacd3-f2d6-4985-8900-6a306cc74ee5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:42.271122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dacd3-f2d6-4985-8900-6a306cc74ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:42.271505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dacd3-f2d6-4985-8900-6a306cc74ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:42.300133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-570dacd3-f2d6-4985-8900-6a306cc74ee5 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:42.301818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-570dacd3-f2d6-4985-8900-6a306cc74ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:42.388003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1133/2301] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:42.389062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f12d59-84e5-4a82-9677-b223758ef48e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:42.404708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f12d59-84e5-4a82-9677-b223758ef48e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:42.410120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:42.410749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:42.411548 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:42.411945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:42.413034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:42.414463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:42.416408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:42.436042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1169/2302] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:42.441567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570dacd3-f2d6-4985-8900-6a306cc74ee5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:42.452301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570dacd3-f2d6-4985-8900-6a306cc74ee5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:42.457577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a338a-418f-4468-905a-555316e97f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:42.458243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a338a-418f-4468-905a-555316e97f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:42.458516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a338a-418f-4468-905a-555316e97f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:42.458785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a338a-418f-4468-905a-555316e97f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:42.459423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a338a-418f-4468-905a-555316e97f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:42.461363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a338a-418f-4468-905a-555316e97f0e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:42.510631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:42.515620 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1134/2303] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:42 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:42.525072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b823d65-ecf4-413e-9e70-45a491e940e5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:42.604040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:42.604700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:42.604980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:42.605254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:42.605913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:42.608018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:42.635638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1170/2304] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:42 2022] POST /identity/v3/users => generated 285 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:42.639717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a338a-418f-4468-905a-555316e97f0e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:42.652815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:42.655163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:42.663943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a338a-418f-4468-905a-555316e97f0e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:42.703999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b8c7c-6a86-44fe-9dbd-c6db3db22e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:42.706984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b8c7c-6a86-44fe-9dbd-c6db3db22e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:42.709143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b8c7c-6a86-44fe-9dbd-c6db3db22e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:42.709973 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b8c7c-6a86-44fe-9dbd-c6db3db22e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:42.711383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cdf92-855d-4320-8550-e6126909435e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:42.711979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b8c7c-6a86-44fe-9dbd-c6db3db22e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:42.714558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b8c7c-6a86-44fe-9dbd-c6db3db22e78 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:42.718071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:42.718729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:42.718998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:42.719259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] PATH_INFO: `/v3/projects/65a43c9033384f73b262335dbf9e1544` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:42.720656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a43c9033384f73b262335dbf9e1544)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:42.723008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:42.725665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:42.759370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1135/2305] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:42.767741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b8c7c-6a86-44fe-9dbd-c6db3db22e78 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:42.802543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a43c9033384f73b262335dbf9e1544', 'request_id': 'req-eb059a06-23a6-46e9-89db-7504d006a2a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.814712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b8c7c-6a86-44fe-9dbd-c6db3db22e78 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:42.826458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a43c9033384f73b262335dbf9e1544', 'request_id': 'req-eb059a06-23a6-46e9-89db-7504d006a2a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.853452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a43c9033384f73b262335dbf9e1544', 'request_id': 'req-eb059a06-23a6-46e9-89db-7504d006a2a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.884591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] The token cache is being invalidate because project 65a43c9033384f73b262335dbf9e1544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:42.884893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.887559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:42.893379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1171/2306] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:42 2022] DELETE /identity/v3/projects/65a43c9033384f73b262335dbf9e1544 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:42.898561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:42.901893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:42.950042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:42.951091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:42.952802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:42.953262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:42.954123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:42.957371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:42.984022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1136/2307] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:42.984980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:42.988033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:43.011545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffd521-8456-4b39-8e7f-477dbb91fd52 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:43.017390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386c4af-6dd2-4e71-8727-3d92bd7e2407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:43.018194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c4af-6dd2-4e71-8727-3d92bd7e2407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:43.018550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c4af-6dd2-4e71-8727-3d92bd7e2407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:43.018841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386c4af-6dd2-4e71-8727-3d92bd7e2407 admin admin] PATH_INFO: `/v3/projects/18c33686414c4b59b27596846e6c233c/users/cb6901e7eaa846fbb84d40124f2868c7/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:43.034179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb059a06-23a6-46e9-89db-7504d006a2a9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:43.041401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386c4af-6dd2-4e71-8727-3d92bd7e2407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c33686414c4b59b27596846e6c233c, user_id=cb6901e7eaa846fbb84d40124f2868c7, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:43.041600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:43.042228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:43.042509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:43.042783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] PATH_INFO: `/v3/users/2f1240973c0c410392531b0114740764` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:43.044116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1240973c0c410392531b0114740764)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:43.044605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386c4af-6dd2-4e71-8727-3d92bd7e2407 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:43.046221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:43.056075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:43.065367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1137/2308] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:42 2022] PUT /identity/v3/projects/18c33686414c4b59b27596846e6c233c/users/cb6901e7eaa846fbb84d40124f2868c7/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:43.071694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dec1cb-3128-4b00-b01a-ddc4d9089ede None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:43.072177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dec1cb-3128-4b00-b01a-ddc4d9089ede None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:43.072485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dec1cb-3128-4b00-b01a-ddc4d9089ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:43.096113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e1dec1cb-3128-4b00-b01a-ddc4d9089ede None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:43.099831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e1dec1cb-3128-4b00-b01a-ddc4d9089ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:43.111674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1240973c0c410392531b0114740764'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.121661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1240973c0c410392531b0114740764'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.137005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1240973c0c410392531b0114740764'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.137752 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1240973c0c410392531b0114740764'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.138401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1240973c0c410392531b0114740764'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.161221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1172/2309] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:15:42 2022] DELETE /identity/v3/users/2f1240973c0c410392531b0114740764 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:43.191717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1138/2310] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:43.203352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:43.204842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:43.223606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:43.292562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fc055-5230-40d3-a5be-1aa5060f5c38 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:43.296897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f531cc-2588-469e-9aa1-367cbbe99f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:43.297623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f531cc-2588-469e-9aa1-367cbbe99f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:43.297949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f531cc-2588-469e-9aa1-367cbbe99f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:43.298692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f531cc-2588-469e-9aa1-367cbbe99f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:43.402329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f531cc-2588-469e-9aa1-367cbbe99f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:43.404747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f531cc-2588-469e-9aa1-367cbbe99f1c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:43.420332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1173/2311] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:43.773874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dec1cb-3128-4b00-b01a-ddc4d9089ede None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:43.776645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e1dec1cb-3128-4b00-b01a-ddc4d9089ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:43.795505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dec1cb-3128-4b00-b01a-ddc4d9089ede None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:43.809165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:43.810105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:43.810582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:43.810897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] PATH_INFO: `/v3/projects/e4d4855d8fc24ab2b294a90b4d355b3e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:43.812098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d4855d8fc24ab2b294a90b4d355b3e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:43.814346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:43.892912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d4855d8fc24ab2b294a90b4d355b3e', 'request_id': 'req-792ab86d-78d8-4041-a8ca-77d283d92b45'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.901983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d4855d8fc24ab2b294a90b4d355b3e', 'request_id': 'req-792ab86d-78d8-4041-a8ca-77d283d92b45'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.913104 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d4855d8fc24ab2b294a90b4d355b3e', 'request_id': 'req-792ab86d-78d8-4041-a8ca-77d283d92b45'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.938146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] The token cache is being invalidate because project e4d4855d8fc24ab2b294a90b4d355b3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:15:43.938498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.939158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-792ab86d-78d8-4041-a8ca-77d283d92b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:15:43.942010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1139/2312] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:15:43 2022] DELETE /identity/v3/projects/e4d4855d8fc24ab2b294a90b4d355b3e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:43.948803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02a53-5fd3-461e-bb45-1282396fe541 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:43.950082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02a53-5fd3-461e-bb45-1282396fe541 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:43.950576 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f02a53-5fd3-461e-bb45-1282396fe541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.002477 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f0f02a53-5fd3-461e-bb45-1282396fe541 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:44.007493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f0f02a53-5fd3-461e-bb45-1282396fe541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:44.093194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1174/2313] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:44.101866 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e954-cbb4-458d-ab9b-9a41d162a1a8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.102193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e954-cbb4-458d-ab9b-9a41d162a1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.102614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4e954-cbb4-458d-ab9b-9a41d162a1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.144396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c7f4e954-cbb4-458d-ab9b-9a41d162a1a8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:44.146240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c7f4e954-cbb4-458d-ab9b-9a41d162a1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:44.154651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1140/2314] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:44.159090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f02a53-5fd3-461e-bb45-1282396fe541 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.169795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f02a53-5fd3-461e-bb45-1282396fe541 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.176212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.178000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.178290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.178770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.180063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:15:44.180495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.183044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.201064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1175/2315] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:15:44.205338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e954-cbb4-458d-ab9b-9a41d162a1a8 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.216599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4e954-cbb4-458d-ab9b-9a41d162a1a8 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.221971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c0310-19a4-48c7-ba27-ef9d63daaf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.222661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0310-19a4-48c7-ba27-ef9d63daaf97 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.222943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0310-19a4-48c7-ba27-ef9d63daaf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.223272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0310-19a4-48c7-ba27-ef9d63daaf97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.223909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c0310-19a4-48c7-ba27-ef9d63daaf97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.226139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c0310-19a4-48c7-ba27-ef9d63daaf97 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.260277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1141/2316] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:44 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:15:44.264500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.274948 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9386493-d21c-496e-bdf9-f71bac2b1794 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.331859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:44.346507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f50e0-116d-4591-a6a5-a7da4b964ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.347222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f50e0-116d-4591-a6a5-a7da4b964ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.347546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f50e0-116d-4591-a6a5-a7da4b964ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.347819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f50e0-116d-4591-a6a5-a7da4b964ee2 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.348692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f50e0-116d-4591-a6a5-a7da4b964ee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.350650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f50e0-116d-4591-a6a5-a7da4b964ee2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.359474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b8616-f8d2-43e6-9f8c-dd85d9c1beb3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.359766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b8616-f8d2-43e6-9f8c-dd85d9c1beb3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.360047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b8616-f8d2-43e6-9f8c-dd85d9c1beb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.376519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1176/2317] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:44 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:44.381551 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f50e0-116d-4591-a6a5-a7da4b964ee2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.395237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f50e0-116d-4591-a6a5-a7da4b964ee2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.402329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281f21-b99e-49ac-99d2-92d5a1ac5210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.403177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281f21-b99e-49ac-99d2-92d5a1ac5210 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.403451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281f21-b99e-49ac-99d2-92d5a1ac5210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.403986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f281f21-b99e-49ac-99d2-92d5a1ac5210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.404661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f281f21-b99e-49ac-99d2-92d5a1ac5210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.406190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5f8b8616-f8d2-43e6-9f8c-dd85d9c1beb3 None None] MFA Rules not processed for user `cb6901e7eaa846fbb84d40124f2868c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:44.406412 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f281f21-b99e-49ac-99d2-92d5a1ac5210 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.408923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5f8b8616-f8d2-43e6-9f8c-dd85d9c1beb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:44.421111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1177/2318] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:44.422267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281f21-b99e-49ac-99d2-92d5a1ac5210 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.434460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f281f21-b99e-49ac-99d2-92d5a1ac5210 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.440051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe2146-6f4a-4909-ab46-4167eddbdb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.443421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2146-6f4a-4909-ab46-4167eddbdb2e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.443421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2146-6f4a-4909-ab46-4167eddbdb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.443421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2146-6f4a-4909-ab46-4167eddbdb2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.443421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe2146-6f4a-4909-ab46-4167eddbdb2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.444257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe2146-6f4a-4909-ab46-4167eddbdb2e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.456708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1178/2319] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:15:44.465327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:44.467131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe2146-6f4a-4909-ab46-4167eddbdb2e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.479791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe2146-6f4a-4909-ab46-4167eddbdb2e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.485360 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133b74f-2b6f-4aa0-843a-271a81285d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.486054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133b74f-2b6f-4aa0-843a-271a81285d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.486341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133b74f-2b6f-4aa0-843a-271a81285d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.486619 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133b74f-2b6f-4aa0-843a-271a81285d8a admin admin] PATH_INFO: `/v3/projects/8e8930f870864f90b74da0248c646eb9/users/75310a6eb1da4347932604faec663801/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.506648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1142/2320] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:44.511508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b8616-f8d2-43e6-9f8c-dd85d9c1beb3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.511822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133b74f-2b6f-4aa0-843a-271a81285d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8930f870864f90b74da0248c646eb9, user_id=75310a6eb1da4347932604faec663801, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.513440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5f8b8616-f8d2-43e6-9f8c-dd85d9c1beb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:44.515260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133b74f-2b6f-4aa0-843a-271a81285d8a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.529385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b8616-f8d2-43e6-9f8c-dd85d9c1beb3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.531765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1179/2321] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:44 2022] PUT /identity/v3/projects/8e8930f870864f90b74da0248c646eb9/users/75310a6eb1da4347932604faec663801/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:44.544597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28c28e-fc5a-4c48-85fb-067f54796518 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.545038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28c28e-fc5a-4c48-85fb-067f54796518 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.545249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28c28e-fc5a-4c48-85fb-067f54796518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.594881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bf28c28e-fc5a-4c48-85fb-067f54796518 None None] MFA Rules not processed for user `75310a6eb1da4347932604faec663801`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:44.596852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bf28c28e-fc5a-4c48-85fb-067f54796518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:44.619114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506ae0e-66f5-46c1-8e97-9a1a7646b3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.620075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ae0e-66f5-46c1-8e97-9a1a7646b3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.620564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ae0e-66f5-46c1-8e97-9a1a7646b3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.621013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ae0e-66f5-46c1-8e97-9a1a7646b3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.622063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506ae0e-66f5-46c1-8e97-9a1a7646b3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.625302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506ae0e-66f5-46c1-8e97-9a1a7646b3e1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.677558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1143/2322] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:15:44 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:44.681866 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506ae0e-66f5-46c1-8e97-9a1a7646b3e1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.705835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1180/2323] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:15:44.720210 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28c28e-fc5a-4c48-85fb-067f54796518 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.722317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bf28c28e-fc5a-4c48-85fb-067f54796518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:44.785108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506ae0e-66f5-46c1-8e97-9a1a7646b3e1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.794013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6975e3-7c20-4832-b0b8-9d305dc1a547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.794648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6975e3-7c20-4832-b0b8-9d305dc1a547 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.794964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6975e3-7c20-4832-b0b8-9d305dc1a547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.795352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6975e3-7c20-4832-b0b8-9d305dc1a547 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.796029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6975e3-7c20-4832-b0b8-9d305dc1a547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.799607 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6975e3-7c20-4832-b0b8-9d305dc1a547 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.818279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28c28e-fc5a-4c48-85fb-067f54796518 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.843297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1144/2324] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:15:44 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:15:44.846839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6975e3-7c20-4832-b0b8-9d305dc1a547 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.864976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6975e3-7c20-4832-b0b8-9d305dc1a547 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.870687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:44.871688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ce5c-e576-4e28-aa75-e23ca4ba7b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.872419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ce5c-e576-4e28-aa75-e23ca4ba7b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.874204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ce5c-e576-4e28-aa75-e23ca4ba7b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.874532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ce5c-e576-4e28-aa75-e23ca4ba7b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.876052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690ce5c-e576-4e28-aa75-e23ca4ba7b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.877706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690ce5c-e576-4e28-aa75-e23ca4ba7b04 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.894154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5b656-5f91-4cf0-9737-d3e1058ba513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.895148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b656-5f91-4cf0-9737-d3e1058ba513 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.895695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b656-5f91-4cf0-9737-d3e1058ba513 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.896285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5b656-5f91-4cf0-9737-d3e1058ba513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.907297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1145/2325] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:44.908309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ce5c-e576-4e28-aa75-e23ca4ba7b04 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.927501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ce5c-e576-4e28-aa75-e23ca4ba7b04 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:44.937015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c26de-14bf-42b6-9229-152afc30da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:44.937826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c26de-14bf-42b6-9229-152afc30da19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:44.938485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c26de-14bf-42b6-9229-152afc30da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:44.938485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c26de-14bf-42b6-9229-152afc30da19 admin admin] PATH_INFO: `/v3/projects/43b0d2bb46774495bdb6c4e56c49e042/users/901bd8747cc14694bcd128778196bd94/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:44.961769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c26de-14bf-42b6-9229-152afc30da19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b0d2bb46774495bdb6c4e56c49e042, user_id=901bd8747cc14694bcd128778196bd94, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.966205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c26de-14bf-42b6-9229-152afc30da19 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:44.983595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1146/2326] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:44 2022] PUT /identity/v3/projects/43b0d2bb46774495bdb6c4e56c49e042/users/901bd8747cc14694bcd128778196bd94/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:15:44.986386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5b656-5f91-4cf0-9737-d3e1058ba513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:44.988500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c26de-14bf-42b6-9229-152afc30da19 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:44.988738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5b656-5f91-4cf0-9737-d3e1058ba513 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:45.001224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1181/2327] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:45.019083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:45.070488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c26de-14bf-42b6-9229-152afc30da19 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:45.076844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da8511-ef5c-417c-9439-baea171ada65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:45.077595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da8511-ef5c-417c-9439-baea171ada65 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:45.077950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da8511-ef5c-417c-9439-baea171ada65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:45.078367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da8511-ef5c-417c-9439-baea171ada65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:45.079325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da8511-ef5c-417c-9439-baea171ada65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:45.081020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da8511-ef5c-417c-9439-baea171ada65 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:45.095324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1147/2328] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:15:45.099389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5b656-5f91-4cf0-9737-d3e1058ba513 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:45.112709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5b656-5f91-4cf0-9737-d3e1058ba513 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:45.120312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:45.121305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:45.121728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:45.122151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] PATH_INFO: `/v3/projects/43b0d2bb46774495bdb6c4e56c49e042/users/901bd8747cc14694bcd128778196bd94/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:45.124978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b0d2bb46774495bdb6c4e56c49e042, user_id=901bd8747cc14694bcd128778196bd94, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:45.128952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:45.159931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1182/2329] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:15:45 2022] PUT /identity/v3/projects/43b0d2bb46774495bdb6c4e56c49e042/users/901bd8747cc14694bcd128778196bd94/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:15:46.057733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2e329-64bd-4c68-ba8f-f2fb41beb9a8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:46.058262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2e329-64bd-4c68-ba8f-f2fb41beb9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:46.058571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2e329-64bd-4c68-ba8f-f2fb41beb9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:46.090839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6db2e329-64bd-4c68-ba8f-f2fb41beb9a8 None None] MFA Rules not processed for user `901bd8747cc14694bcd128778196bd94`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:15:46.092067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6db2e329-64bd-4c68-ba8f-f2fb41beb9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:46.231855 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1148/2330] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:15:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:15:46.243760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:46.246019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:15:46.307584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5098d5-e3e6-411f-9c78-991dfeb4a1bc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:46.311531 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0eed8-5a48-43a5-aa4a-1711cb467253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:46.312138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0eed8-5a48-43a5-aa4a-1711cb467253 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:46.312506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0eed8-5a48-43a5-aa4a-1711cb467253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:46.312819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0eed8-5a48-43a5-aa4a-1711cb467253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:46.375076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0eed8-5a48-43a5-aa4a-1711cb467253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:46.376581 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0eed8-5a48-43a5-aa4a-1711cb467253 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:46.387080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1183/2331] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:47.938717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2e329-64bd-4c68-ba8f-f2fb41beb9a8 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:15:47.951006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2e329-64bd-4c68-ba8f-f2fb41beb9a8 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:15:47.954814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813bfac-3c58-4ecb-a646-3ed5507a8ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:15:47.955432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bfac-3c58-4ecb-a646-3ed5507a8ecd service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:15:47.955712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bfac-3c58-4ecb-a646-3ed5507a8ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:15:47.955988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bfac-3c58-4ecb-a646-3ed5507a8ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:15:47.966793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813bfac-3c58-4ecb-a646-3ed5507a8ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:15:47.968390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813bfac-3c58-4ecb-a646-3ed5507a8ecd service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:15:47.982478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1149/2332] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:15:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:15:56.382635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:15:57.976684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=59 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:07.376744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0eed8-5a48-43a5-aa4a-1711cb467253 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:07.464759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0eed8-5a48-43a5-aa4a-1711cb467253 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:07.473630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:07.474519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:07.474949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:07.475572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] PATH_INFO: `/v3/users/75310a6eb1da4347932604faec663801` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:07.477140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=75310a6eb1da4347932604faec663801)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:07.479245 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:07.574949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75310a6eb1da4347932604faec663801'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:07.585138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75310a6eb1da4347932604faec663801'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:07.599037 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75310a6eb1da4347932604faec663801'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:07.601029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75310a6eb1da4347932604faec663801'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:07.602187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75310a6eb1da4347932604faec663801'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:07.716978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1184/2333] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:16:07 2022] DELETE /identity/v3/users/75310a6eb1da4347932604faec663801 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:07.720309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c3bc5-8bd7-48bd-8db1-02f1873ee9fb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:07.721431 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c3bc5-8bd7-48bd-8db1-02f1873ee9fb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:07.721879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c3bc5-8bd7-48bd-8db1-02f1873ee9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:07.779975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a92c3bc5-8bd7-48bd-8db1-02f1873ee9fb None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:07.783929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a92c3bc5-8bd7-48bd-8db1-02f1873ee9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:07.947768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1150/2334] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:07.967992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:07.969734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:08.126034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e04f84-5189-4abd-a0e1-1fb2501474cb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:08.133184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f9ffe-e9f9-4775-92e6-f350f24aca13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:08.133920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f9ffe-e9f9-4775-92e6-f350f24aca13 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:08.134298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f9ffe-e9f9-4775-92e6-f350f24aca13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:08.134673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f9ffe-e9f9-4775-92e6-f350f24aca13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:08.145698 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f9ffe-e9f9-4775-92e6-f350f24aca13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:08.147940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f9ffe-e9f9-4775-92e6-f350f24aca13 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:08.160173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1185/2335] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:08.927686 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c3bc5-8bd7-48bd-8db1-02f1873ee9fb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:08.930251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a92c3bc5-8bd7-48bd-8db1-02f1873ee9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:08.954471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c3bc5-8bd7-48bd-8db1-02f1873ee9fb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:08.960186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:08.961652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:08.961839 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:08.962200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] PATH_INFO: `/v3/projects/8e8930f870864f90b74da0248c646eb9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:08.963829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8930f870864f90b74da0248c646eb9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:08.966362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:09.028306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8930f870864f90b74da0248c646eb9', 'request_id': 'req-d98ae59f-d659-4913-b045-54f5d645196f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:09.033695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8930f870864f90b74da0248c646eb9', 'request_id': 'req-d98ae59f-d659-4913-b045-54f5d645196f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:09.042121 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8930f870864f90b74da0248c646eb9', 'request_id': 'req-d98ae59f-d659-4913-b045-54f5d645196f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:09.053326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] The token cache is being invalidate because project 8e8930f870864f90b74da0248c646eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:16:09.053510 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:09.054133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:09.055636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1151/2336] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:16:08 2022] DELETE /identity/v3/projects/8e8930f870864f90b74da0248c646eb9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:12.465091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4dbec-3405-4469-b7ef-52d8bf5ac523 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:12.465615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4dbec-3405-4469-b7ef-52d8bf5ac523 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:12.466751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4dbec-3405-4469-b7ef-52d8bf5ac523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:12.493339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5ee4dbec-3405-4469-b7ef-52d8bf5ac523 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:12.494566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5ee4dbec-3405-4469-b7ef-52d8bf5ac523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:12.517832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:12.570542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1186/2337] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:12.618349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:12.619759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:12.714958 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ae59f-d659-4913-b045-54f5d645196f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:12.721393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff16c0-0746-4954-891a-af0d96e63f17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:12.722010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff16c0-0746-4954-891a-af0d96e63f17 service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:12.722319 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff16c0-0746-4954-891a-af0d96e63f17 service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:12.722596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff16c0-0746-4954-891a-af0d96e63f17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:12.731388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff16c0-0746-4954-891a-af0d96e63f17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:12.733005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff16c0-0746-4954-891a-af0d96e63f17 service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:12.742765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1152/2338] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:16:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:16:14.665390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b7d3-5bb0-468b-87bb-1a7897f7695f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:14.666760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b7d3-5bb0-468b-87bb-1a7897f7695f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:14.667106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342b7d3-5bb0-468b-87bb-1a7897f7695f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:14.669467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1187/2339] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:16:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:16:14.706878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf68f-65df-4ce5-b551-99de71b96439 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:14.707511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf68f-65df-4ce5-b551-99de71b96439 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:14.707858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf68f-65df-4ce5-b551-99de71b96439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:14.731015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b8bbf68f-65df-4ce5-b551-99de71b96439 None None] MFA Rules not processed for user `79078cd57816487e8b2b85856c13d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:14.733694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b8bbf68f-65df-4ce5-b551-99de71b96439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:14.745433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:14.832135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1153/2340] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:16:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:14.873915 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342b7d3-5bb0-468b-87bb-1a7897f7695f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:14.876026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f342b7d3-5bb0-468b-87bb-1a7897f7695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:14.908168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342b7d3-5bb0-468b-87bb-1a7897f7695f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:14.915702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177d2bd-e5c8-4f62-8d7d-24d79dd65a24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:14.916546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177d2bd-e5c8-4f62-8d7d-24d79dd65a24 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:14.916818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177d2bd-e5c8-4f62-8d7d-24d79dd65a24 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:14.917285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177d2bd-e5c8-4f62-8d7d-24d79dd65a24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:14.935092 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177d2bd-e5c8-4f62-8d7d-24d79dd65a24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:14.938713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177d2bd-e5c8-4f62-8d7d-24d79dd65a24 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:14.951115 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1188/2341] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:15.891516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbf68f-65df-4ce5-b551-99de71b96439 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:15.894773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b8bbf68f-65df-4ce5-b551-99de71b96439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:15.988719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbf68f-65df-4ce5-b551-99de71b96439 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:15.993587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:16.001185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:16.001683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:16.002301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] PATH_INFO: `/v3/users/cb6901e7eaa846fbb84d40124f2868c7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:16.003862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb6901e7eaa846fbb84d40124f2868c7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:16.006029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:16.081594 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb6901e7eaa846fbb84d40124f2868c7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.090117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb6901e7eaa846fbb84d40124f2868c7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.097523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb6901e7eaa846fbb84d40124f2868c7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.098174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb6901e7eaa846fbb84d40124f2868c7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.098784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb6901e7eaa846fbb84d40124f2868c7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.117568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1154/2342] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:16:15 2022] DELETE /identity/v3/users/cb6901e7eaa846fbb84d40124f2868c7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:16.320133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177d2bd-e5c8-4f62-8d7d-24d79dd65a24 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:16.337589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177d2bd-e5c8-4f62-8d7d-24d79dd65a24 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:16.413865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:16.414590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:16.414877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:16.415146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] PATH_INFO: `/v3/projects/18c33686414c4b59b27596846e6c233c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:16.416571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18c33686414c4b59b27596846e6c233c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:16.418878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:16.480084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18c33686414c4b59b27596846e6c233c', 'request_id': 'req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.486646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18c33686414c4b59b27596846e6c233c', 'request_id': 'req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.501672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18c33686414c4b59b27596846e6c233c', 'request_id': 'req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.524975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] The token cache is being invalidate because project 18c33686414c4b59b27596846e6c233c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:16:16.525549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.526369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.528026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1189/2343] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:16:16 2022] DELETE /identity/v3/projects/18c33686414c4b59b27596846e6c233c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:16.532223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:16.533961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:16.552757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c147bb-9fd5-4e7e-94db-b65865060cda admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:16.589437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:16.630850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:16.631831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:16.632455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:16.632904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] PATH_INFO: `/v3/users/901bd8747cc14694bcd128778196bd94` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:16.634252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] RBAC: Authorizing `identity:delete_user(user_id=901bd8747cc14694bcd128778196bd94)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:16.636575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:16.709703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '901bd8747cc14694bcd128778196bd94'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.714559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '901bd8747cc14694bcd128778196bd94'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.724391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '901bd8747cc14694bcd128778196bd94'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.725825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '901bd8747cc14694bcd128778196bd94'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.726861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-01ba4c15-25ac-420d-9695-14780057329b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '901bd8747cc14694bcd128778196bd94'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:16.753845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1155/2344] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:16:16 2022] DELETE /identity/v3/users/901bd8747cc14694bcd128778196bd94 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:17.009394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:17.011117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:17.051380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b0225-ed0b-4df6-b379-ac8ee2ee7f00 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:17.183072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:17.226496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:17.227969 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:17.228404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:17.228757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] PATH_INFO: `/v3/projects/43b0d2bb46774495bdb6c4e56c49e042` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:17.230236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b0d2bb46774495bdb6c4e56c49e042)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:17.232464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:17.305584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b0d2bb46774495bdb6c4e56c49e042', 'request_id': 'req-89b5b8ac-f9c9-41e8-8173-d593ad83382d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:17.313612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b0d2bb46774495bdb6c4e56c49e042', 'request_id': 'req-89b5b8ac-f9c9-41e8-8173-d593ad83382d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:17.325226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b0d2bb46774495bdb6c4e56c49e042', 'request_id': 'req-89b5b8ac-f9c9-41e8-8173-d593ad83382d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:17.348224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] The token cache is being invalidate because project 43b0d2bb46774495bdb6c4e56c49e042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:16:17.349136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:17.350329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89b5b8ac-f9c9-41e8-8173-d593ad83382d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:17.353164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1190/2345] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:16:17 2022] DELETE /identity/v3/projects/43b0d2bb46774495bdb6c4e56c49e042 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:17.358162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ec1da-db60-4c2e-82e7-7d949153ca2f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:17.359055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ec1da-db60-4c2e-82e7-7d949153ca2f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:17.359336 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ec1da-db60-4c2e-82e7-7d949153ca2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:17.431335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-995ec1da-db60-4c2e-82e7-7d949153ca2f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:17.432925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-995ec1da-db60-4c2e-82e7-7d949153ca2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:17.550427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1156/2346] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:17.559602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f15c2-257c-4878-8966-df1c4fdc5211 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:17.559894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f15c2-257c-4878-8966-df1c4fdc5211 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:17.560970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f15c2-257c-4878-8966-df1c4fdc5211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:17.593774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c86f15c2-257c-4878-8966-df1c4fdc5211 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:17.595221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c86f15c2-257c-4878-8966-df1c4fdc5211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:17.626371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1191/2347] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:17.630862 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ec1da-db60-4c2e-82e7-7d949153ca2f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:17.648111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ec1da-db60-4c2e-82e7-7d949153ca2f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:17.655210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:17.656541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:17.656942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:17.657557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:17.658692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:16:17.659095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:17.661286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:17.681982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1157/2348] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:16:17.683406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f15c2-257c-4878-8966-df1c4fdc5211 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:17.695064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f15c2-257c-4878-8966-df1c4fdc5211 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:17.701416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5196afc-c0f9-486e-a2c3-2a5f7dfbdea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:17.702031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196afc-c0f9-486e-a2c3-2a5f7dfbdea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:17.702305 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196afc-c0f9-486e-a2c3-2a5f7dfbdea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:17.702592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196afc-c0f9-486e-a2c3-2a5f7dfbdea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:17.703390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5196afc-c0f9-486e-a2c3-2a5f7dfbdea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:17.707141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5196afc-c0f9-486e-a2c3-2a5f7dfbdea9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:17.744110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1192/2349] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:16:17 2022] POST /identity/v3/projects => generated 361 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:16:17.748384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:17.765154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59997bd7-8b4f-4586-b430-6617cf9941fa admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:17.814462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:17.855969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c2a23-3b29-4105-8411-baf57555f4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:17.856639 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c2a23-3b29-4105-8411-baf57555f4ad admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:17.856924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c2a23-3b29-4105-8411-baf57555f4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:17.857200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c2a23-3b29-4105-8411-baf57555f4ad admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:17.857844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c2a23-3b29-4105-8411-baf57555f4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:17.859788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c2a23-3b29-4105-8411-baf57555f4ad admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:17.901061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1158/2350] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:16:17 2022] POST /identity/v3/users => generated 281 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:16:17.905526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5196afc-c0f9-486e-a2c3-2a5f7dfbdea9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:17.936397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5196afc-c0f9-486e-a2c3-2a5f7dfbdea9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:17.956521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a456c-510a-4e60-8a45-f9574f96950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:17.956521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a456c-510a-4e60-8a45-f9574f96950e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:17.956521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a456c-510a-4e60-8a45-f9574f96950e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:17.956521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a456c-510a-4e60-8a45-f9574f96950e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:17.956991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a456c-510a-4e60-8a45-f9574f96950e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:17.956991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a456c-510a-4e60-8a45-f9574f96950e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:17.971138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1193/2351] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:16:17.973997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c2a23-3b29-4105-8411-baf57555f4ad admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:17.995358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c2a23-3b29-4105-8411-baf57555f4ad admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:18.004395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c9f01-f7ae-4f68-8193-a2a998453792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:18.005469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9f01-f7ae-4f68-8193-a2a998453792 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:18.005469 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9f01-f7ae-4f68-8193-a2a998453792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:18.005683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c9f01-f7ae-4f68-8193-a2a998453792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:18.006376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c9f01-f7ae-4f68-8193-a2a998453792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:18.009003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c9f01-f7ae-4f68-8193-a2a998453792 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:18.059829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1159/2352] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:16:18.079951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a456c-510a-4e60-8a45-f9574f96950e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:18.092481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a456c-510a-4e60-8a45-f9574f96950e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:18.097435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:18.098104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:18.098365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:18.098679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] PATH_INFO: `/v3/projects/e99f2421479140cab1fb100b9254cab7/users/1ba412d0c5344396bc132ee25d83e929/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:18.122195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e99f2421479140cab1fb100b9254cab7, user_id=1ba412d0c5344396bc132ee25d83e929, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:18.127595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:18.162274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1194/2353] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:16:18 2022] PUT /identity/v3/projects/e99f2421479140cab1fb100b9254cab7/users/1ba412d0c5344396bc132ee25d83e929/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:18.226288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d28fe4-8a45-4b91-b363-f80604a4e52f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:18.226671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d28fe4-8a45-4b91-b363-f80604a4e52f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:18.226853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d28fe4-8a45-4b91-b363-f80604a4e52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:18.261222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-25d28fe4-8a45-4b91-b363-f80604a4e52f None None] MFA Rules not processed for user `1ba412d0c5344396bc132ee25d83e929`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:18.262648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-25d28fe4-8a45-4b91-b363-f80604a4e52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:18.377757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1160/2354] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:18.427429 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:18.429995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:18.527852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373e80f-3079-4ea4-86c0-23b04ddb01d8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:18.533209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdaf275-46b3-45f4-b0cf-d766b208e827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:18.534653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaf275-46b3-45f4-b0cf-d766b208e827 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:18.535156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaf275-46b3-45f4-b0cf-d766b208e827 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:18.535507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaf275-46b3-45f4-b0cf-d766b208e827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:18.556661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdaf275-46b3-45f4-b0cf-d766b208e827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:18.558247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdaf275-46b3-45f4-b0cf-d766b208e827 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:18.582842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1195/2355] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:18.601052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d28fe4-8a45-4b91-b363-f80604a4e52f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:18.695328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d28fe4-8a45-4b91-b363-f80604a4e52f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:18.700793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:18.701448 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:18.701719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:18.702032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] PATH_INFO: `/v3/users/1ba412d0c5344396bc132ee25d83e929` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:18.703160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ba412d0c5344396bc132ee25d83e929)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:18.705151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:18.775213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:18.780834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ba412d0c5344396bc132ee25d83e929'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:18.794853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ba412d0c5344396bc132ee25d83e929'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:18.804355 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ba412d0c5344396bc132ee25d83e929'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:18.805139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ba412d0c5344396bc132ee25d83e929'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:18.805851 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d53d91ea-8c1d-4992-be8d-2859d6fed5c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ba412d0c5344396bc132ee25d83e929'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:18.836498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1161/2356] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:16:18 2022] DELETE /identity/v3/users/1ba412d0c5344396bc132ee25d83e929 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:18.841686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae57b-bc38-4f8e-b409-0caee7689339 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:18.842165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae57b-bc38-4f8e-b409-0caee7689339 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:18.842642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eae57b-bc38-4f8e-b409-0caee7689339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:18.868052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-94eae57b-bc38-4f8e-b409-0caee7689339 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:18.870154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-94eae57b-bc38-4f8e-b409-0caee7689339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:18.880522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:18.951962 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1713c9-401d-4d70-92a3-eafad3f8076b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:18.952770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1713c9-401d-4d70-92a3-eafad3f8076b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:18.953369 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1713c9-401d-4d70-92a3-eafad3f8076b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:18.969574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1196/2357] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:18.970802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eae57b-bc38-4f8e-b409-0caee7689339 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:18.972959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-94eae57b-bc38-4f8e-b409-0caee7689339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:19.017950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2a1713c9-401d-4d70-92a3-eafad3f8076b None None] MFA Rules not processed for user `8755b16a18734175b691f891351d5116`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:19.023467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2a1713c9-401d-4d70-92a3-eafad3f8076b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:19.075985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eae57b-bc38-4f8e-b409-0caee7689339 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:19.086638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:19.087273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:19.087554 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:19.087822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:19.104421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:19.105953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:19.132570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1197/2358] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:19.252563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1162/2359] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:19.552469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:19.556331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:19.576259 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6655ce-d545-49a5-9d0c-eb395792ab34 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:19.582718 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:19.583690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:19.585079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:19.596720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] PATH_INFO: `/v3/projects/e99f2421479140cab1fb100b9254cab7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:19.598960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e99f2421479140cab1fb100b9254cab7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:19.602233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:19.675433 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e99f2421479140cab1fb100b9254cab7', 'request_id': 'req-0a62f463-e16e-45ac-862a-db738fcebb93'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:19.684956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e99f2421479140cab1fb100b9254cab7', 'request_id': 'req-0a62f463-e16e-45ac-862a-db738fcebb93'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:19.697441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e99f2421479140cab1fb100b9254cab7', 'request_id': 'req-0a62f463-e16e-45ac-862a-db738fcebb93'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:19.716138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] The token cache is being invalidate because project e99f2421479140cab1fb100b9254cab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:16:19.716943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:19.717790 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0a62f463-e16e-45ac-862a-db738fcebb93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:19.719804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1198/2360] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:16:19 2022] DELETE /identity/v3/projects/e99f2421479140cab1fb100b9254cab7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:19.726493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41803-d277-427b-a613-1e76f70ae731 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:19.727032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41803-d277-427b-a613-1e76f70ae731 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:19.727032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41803-d277-427b-a613-1e76f70ae731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:19.751610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-05d41803-d277-427b-a613-1e76f70ae731 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:19.753228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-05d41803-d277-427b-a613-1e76f70ae731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:19.857178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1163/2361] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:19.868680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907124b-fdbf-431e-bd7f-357878325ebd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:19.869143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907124b-fdbf-431e-bd7f-357878325ebd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:19.869492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907124b-fdbf-431e-bd7f-357878325ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:19.898972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d907124b-fdbf-431e-bd7f-357878325ebd None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:19.900683 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d907124b-fdbf-431e-bd7f-357878325ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:19.909521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1199/2362] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:19.914798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41803-d277-427b-a613-1e76f70ae731 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:19.924957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41803-d277-427b-a613-1e76f70ae731 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:19.930845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:19.931582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:19.932074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:19.932456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:19.933210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:16:19.933718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:19.935716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:19.950432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1164/2363] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:16:19.955014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907124b-fdbf-431e-bd7f-357878325ebd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:19.966187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907124b-fdbf-431e-bd7f-357878325ebd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:19.971950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73896f39-54ab-4cda-863c-4da77e464946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:19.972658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896f39-54ab-4cda-863c-4da77e464946 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:19.972934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896f39-54ab-4cda-863c-4da77e464946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:19.973198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896f39-54ab-4cda-863c-4da77e464946 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:19.973879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73896f39-54ab-4cda-863c-4da77e464946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:19.977125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73896f39-54ab-4cda-863c-4da77e464946 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:20.012474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1200/2364] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:16:19 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:16:20.015535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:20.027889 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e854a-1bed-4a65-8e3d-c406977090c0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:20.090850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:20.102285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a3084-ffd5-4b2e-b5b4-0b67954aab9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:20.103601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a3084-ffd5-4b2e-b5b4-0b67954aab9f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:20.104263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a3084-ffd5-4b2e-b5b4-0b67954aab9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:20.104780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a3084-ffd5-4b2e-b5b4-0b67954aab9f admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:20.105650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a3084-ffd5-4b2e-b5b4-0b67954aab9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:20.107667 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a3084-ffd5-4b2e-b5b4-0b67954aab9f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:20.136959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1165/2365] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:16:20 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:16:20.142105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73896f39-54ab-4cda-863c-4da77e464946 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:20.155775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73896f39-54ab-4cda-863c-4da77e464946 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:20.160647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078aaf3-f0fc-46a5-bf27-daabf2b61d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:20.161298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078aaf3-f0fc-46a5-bf27-daabf2b61d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:20.161638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078aaf3-f0fc-46a5-bf27-daabf2b61d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:20.161981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078aaf3-f0fc-46a5-bf27-daabf2b61d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:20.162938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078aaf3-f0fc-46a5-bf27-daabf2b61d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:20.164661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078aaf3-f0fc-46a5-bf27-daabf2b61d7a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:20.171120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:20.183451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1201/2366] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:16:20.185302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a3084-ffd5-4b2e-b5b4-0b67954aab9f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:20.198392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a3084-ffd5-4b2e-b5b4-0b67954aab9f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:20.207001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cce519-bfc3-4442-858b-1f3ba2a0d080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:20.207916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce519-bfc3-4442-858b-1f3ba2a0d080 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:20.208199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce519-bfc3-4442-858b-1f3ba2a0d080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:20.208516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce519-bfc3-4442-858b-1f3ba2a0d080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:20.209893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cce519-bfc3-4442-858b-1f3ba2a0d080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:20.211789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cce519-bfc3-4442-858b-1f3ba2a0d080 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:20.227820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1166/2367] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:16:20.230188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078aaf3-f0fc-46a5-bf27-daabf2b61d7a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:20.254751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078aaf3-f0fc-46a5-bf27-daabf2b61d7a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:20.261695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:20.262470 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:20.262806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:20.263688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] PATH_INFO: `/v3/projects/7dba8b855b394da79f399bb1155f46d1/users/c56286315ef74770af05d02629c525ba/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:20.281237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dba8b855b394da79f399bb1155f46d1, user_id=c56286315ef74770af05d02629c525ba, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:20.283978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:20.299597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1202/2368] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:16:20 2022] PUT /identity/v3/projects/7dba8b855b394da79f399bb1155f46d1/users/c56286315ef74770af05d02629c525ba/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:20.310505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc54a-b1f5-4ce1-850f-31d4952cc45e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:20.310780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc54a-b1f5-4ce1-850f-31d4952cc45e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:20.311050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bc54a-b1f5-4ce1-850f-31d4952cc45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:20.347426 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-442bc54a-b1f5-4ce1-850f-31d4952cc45e None None] MFA Rules not processed for user `c56286315ef74770af05d02629c525ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:20.349255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-442bc54a-b1f5-4ce1-850f-31d4952cc45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:20.458359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1167/2369] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:20.474280 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:20.478110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:20.572918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925eb3c-0bd8-43fc-b66e-4767f60feec5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:20.578159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:20.578872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:20.579228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:20.579508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:20.597396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:20.601292 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:20.611570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1203/2370] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:30.452509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:30.605852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:33.439373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac1cd4-dca2-4ae8-b422-df57cf731396 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:33.439885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac1cd4-dca2-4ae8-b422-df57cf731396 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:33.439885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac1cd4-dca2-4ae8-b422-df57cf731396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:33.466289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-91ac1cd4-dca2-4ae8-b422-df57cf731396 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:33.467796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-91ac1cd4-dca2-4ae8-b422-df57cf731396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:33.546850 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1168/2371] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:16:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:33.561029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:33.562660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:33.642190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03581bf4-4417-468b-86ff-0a43145ca1f5 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:33.650605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bf61a-9c8f-48b0-ae4e-f2ed049dc707 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:33.651981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bf61a-9c8f-48b0-ae4e-f2ed049dc707 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:33.652355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bf61a-9c8f-48b0-ae4e-f2ed049dc707 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:33.652754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bf61a-9c8f-48b0-ae4e-f2ed049dc707 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:33.661986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bf61a-9c8f-48b0-ae4e-f2ed049dc707 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:33.665301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bf61a-9c8f-48b0-ae4e-f2ed049dc707 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:33.674783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1204/2372] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:16:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:16:34.317825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac1cd4-dca2-4ae8-b422-df57cf731396 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:34.582996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac1cd4-dca2-4ae8-b422-df57cf731396 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:34.610677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:34.613457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:34.617531 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:34.618452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] PATH_INFO: `/v3/users/c56286315ef74770af05d02629c525ba` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:34.622996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56286315ef74770af05d02629c525ba)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:34.628114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:34.785006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56286315ef74770af05d02629c525ba'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:34.806227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56286315ef74770af05d02629c525ba'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:34.825658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56286315ef74770af05d02629c525ba'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:34.829013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56286315ef74770af05d02629c525ba'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:34.829013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56286315ef74770af05d02629c525ba'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:34.882451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1169/2373] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:16:34 2022] DELETE /identity/v3/users/c56286315ef74770af05d02629c525ba => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:34.883589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9186-f8f7-40a5-a9fb-0d370ced232f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:34.884122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9186-f8f7-40a5-a9fb-0d370ced232f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:34.884122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9186-f8f7-40a5-a9fb-0d370ced232f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:34.950553 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-14cf9186-f8f7-40a5-a9fb-0d370ced232f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:34.951899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-14cf9186-f8f7-40a5-a9fb-0d370ced232f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:35.059456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1205/2374] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:35.144719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:35.147138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:35.223017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:35.302347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb389b1-36a2-4513-8219-c2ac66203997 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:35.319161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540aaee-2531-47bf-8818-86d9a8318c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:35.319902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540aaee-2531-47bf-8818-86d9a8318c0c service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:35.320288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540aaee-2531-47bf-8818-86d9a8318c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:35.320745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540aaee-2531-47bf-8818-86d9a8318c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:35.339223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540aaee-2531-47bf-8818-86d9a8318c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:35.340930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540aaee-2531-47bf-8818-86d9a8318c0c service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:35.362006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1170/2375] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:36.194846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf9186-f8f7-40a5-a9fb-0d370ced232f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:36.199134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-14cf9186-f8f7-40a5-a9fb-0d370ced232f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:36.277990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf9186-f8f7-40a5-a9fb-0d370ced232f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:36.297423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:36.298081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:36.298618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:36.298879 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] PATH_INFO: `/v3/projects/7dba8b855b394da79f399bb1155f46d1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:36.300176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dba8b855b394da79f399bb1155f46d1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:36.308015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:36.450066 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dba8b855b394da79f399bb1155f46d1', 'request_id': 'req-1a518396-d0d9-4c52-ba4f-38766c824944'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:36.458716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dba8b855b394da79f399bb1155f46d1', 'request_id': 'req-1a518396-d0d9-4c52-ba4f-38766c824944'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:36.475196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dba8b855b394da79f399bb1155f46d1', 'request_id': 'req-1a518396-d0d9-4c52-ba4f-38766c824944'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:36.497678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] The token cache is being invalidate because project 7dba8b855b394da79f399bb1155f46d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:16:36.498437 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:36.499420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:36.504047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1206/2376] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:16:36 2022] DELETE /identity/v3/projects/7dba8b855b394da79f399bb1155f46d1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:42.269657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c635f4-3dbe-445d-8023-48dd1b2e287e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:42.270151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c635f4-3dbe-445d-8023-48dd1b2e287e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:42.270261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c635f4-3dbe-445d-8023-48dd1b2e287e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:42.305846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c2c635f4-3dbe-445d-8023-48dd1b2e287e None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:42.307150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c2c635f4-3dbe-445d-8023-48dd1b2e287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:42.398617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1171/2377] 10.222.0.70 () {58 vars in 967 bytes} [Tue Aug 9 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:42.411297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:42.412786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:42.429394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:42.555714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a518396-d0d9-4c52-ba4f-38766c824944 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:42.562038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4b03c-3975-44ac-ae3c-4746176a0def service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:42.562747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4b03c-3975-44ac-ae3c-4746176a0def service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:42.563213 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4b03c-3975-44ac-ae3c-4746176a0def service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:42.563555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4b03c-3975-44ac-ae3c-4746176a0def service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:42.573625 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4b03c-3975-44ac-ae3c-4746176a0def service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:42.575476 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4b03c-3975-44ac-ae3c-4746176a0def service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:42.585327 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1207/2378] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:16:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 19:16:44.217558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c635f4-3dbe-445d-8023-48dd1b2e287e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:44.219302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c2c635f4-3dbe-445d-8023-48dd1b2e287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:44.306222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c635f4-3dbe-445d-8023-48dd1b2e287e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:44.313221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:44.314022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:44.314364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:44.314708 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] PATH_INFO: `/v3/users/deca408116c841cdbfbe0a83ca240c5e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:44.315961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=deca408116c841cdbfbe0a83ca240c5e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:44.318676 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:44.378258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deca408116c841cdbfbe0a83ca240c5e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:44.403587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deca408116c841cdbfbe0a83ca240c5e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:44.413825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deca408116c841cdbfbe0a83ca240c5e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:44.414797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deca408116c841cdbfbe0a83ca240c5e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:44.415475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deca408116c841cdbfbe0a83ca240c5e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:44.432482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:44.451652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434784-85c7-465e-9008-cacc020c9aaa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:44.452550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434784-85c7-465e-9008-cacc020c9aaa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:44.452982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434784-85c7-465e-9008-cacc020c9aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:44.456838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1172/2379] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:16:44 2022] DELETE /identity/v3/users/deca408116c841cdbfbe0a83ca240c5e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:44.476261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-48434784-85c7-465e-9008-cacc020c9aaa None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:44.477964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-48434784-85c7-465e-9008-cacc020c9aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:44.558540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1208/2380] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:44.569275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:44.570933 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:44.647951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965bf7-6d45-4937-ad2a-36b3211147ad admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:44.667905 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513149e5-a726-4c79-8850-94ca72ca4f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:44.668902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513149e5-a726-4c79-8850-94ca72ca4f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:44.669184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513149e5-a726-4c79-8850-94ca72ca4f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:44.669752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513149e5-a726-4c79-8850-94ca72ca4f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:44.683365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513149e5-a726-4c79-8850-94ca72ca4f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:44.685880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513149e5-a726-4c79-8850-94ca72ca4f25 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:44.699013 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1173/2381] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:45.108341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48434784-85c7-465e-9008-cacc020c9aaa None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.109871 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-48434784-85c7-465e-9008-cacc020c9aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:45.138934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48434784-85c7-465e-9008-cacc020c9aaa None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.150537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:45.151240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.151522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.151983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] PATH_INFO: `/v3/projects/f524447f3a87499f804b0cd2acf58cc9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.153250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f524447f3a87499f804b0cd2acf58cc9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:45.155174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:45.232936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f524447f3a87499f804b0cd2acf58cc9', 'request_id': 'req-26528a87-eaa8-4b05-8224-d45911423456'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.241056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f524447f3a87499f804b0cd2acf58cc9', 'request_id': 'req-26528a87-eaa8-4b05-8224-d45911423456'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.254028 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f524447f3a87499f804b0cd2acf58cc9', 'request_id': 'req-26528a87-eaa8-4b05-8224-d45911423456'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.268363 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] The token cache is being invalidate because project f524447f3a87499f804b0cd2acf58cc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:16:45.268876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.269556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26528a87-eaa8-4b05-8224-d45911423456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.275152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1209/2382] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:16:45 2022] DELETE /identity/v3/projects/f524447f3a87499f804b0cd2acf58cc9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:45.277074 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973aa6f-2389-4caa-8bec-b54bb075558d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.277548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973aa6f-2389-4caa-8bec-b54bb075558d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.277957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973aa6f-2389-4caa-8bec-b54bb075558d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.308421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2973aa6f-2389-4caa-8bec-b54bb075558d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:45.308421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2973aa6f-2389-4caa-8bec-b54bb075558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:45.405627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1174/2383] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:45.414420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47119b-c87e-44b0-bf6f-02e4e4ebadf4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.414640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47119b-c87e-44b0-bf6f-02e4e4ebadf4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.415064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47119b-c87e-44b0-bf6f-02e4e4ebadf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.440788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3a47119b-c87e-44b0-bf6f-02e4e4ebadf4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:45.443010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3a47119b-c87e-44b0-bf6f-02e4e4ebadf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:45.455916 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1210/2384] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:45.462251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973aa6f-2389-4caa-8bec-b54bb075558d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.492032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973aa6f-2389-4caa-8bec-b54bb075558d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.506264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:45.506264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.506264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.506264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.509249 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47119b-c87e-44b0-bf6f-02e4e4ebadf4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.511749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3a47119b-c87e-44b0-bf6f-02e4e4ebadf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:45.513483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:16:45.513908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:45.515747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:45.531394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47119b-c87e-44b0-bf6f-02e4e4ebadf4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.533422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1175/2385] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:16:45.534613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.545349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:45.546493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.546776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.547049 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] PATH_INFO: `/v3/users/eef1ea8d331444e892d0b30c1feef904` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.547436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccbfe9-208f-42dd-b2e6-426fc5820ebd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.549422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef1ea8d331444e892d0b30c1feef904)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:45.551546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:45.554658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a3b0-9d86-4a26-bce9-638b235276dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:45.555599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a3b0-9d86-4a26-bce9-638b235276dd admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.556036 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a3b0-9d86-4a26-bce9-638b235276dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.556516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a3b0-9d86-4a26-bce9-638b235276dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.557871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a3b0-9d86-4a26-bce9-638b235276dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:45.560313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a3b0-9d86-4a26-bce9-638b235276dd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:45.575174 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:45.604923 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1176/2386] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:16:45 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:16:45.609140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a3b0-9d86-4a26-bce9-638b235276dd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.633571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef1ea8d331444e892d0b30c1feef904'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.644320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef1ea8d331444e892d0b30c1feef904'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.662724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef1ea8d331444e892d0b30c1feef904'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.664643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef1ea8d331444e892d0b30c1feef904'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.665616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef1ea8d331444e892d0b30c1feef904'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:45.716502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1211/2387] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:16:45 2022] DELETE /identity/v3/users/eef1ea8d331444e892d0b30c1feef904 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:45.731846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a3b0-9d86-4a26-bce9-638b235276dd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.745580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:45.845643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f52738-5b8b-4d69-a146-8f1e9b2490ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:45.846262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52738-5b8b-4d69-a146-8f1e9b2490ab admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.846542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52738-5b8b-4d69-a146-8f1e9b2490ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.846881 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52738-5b8b-4d69-a146-8f1e9b2490ab admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.847858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f52738-5b8b-4d69-a146-8f1e9b2490ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:45.850459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f52738-5b8b-4d69-a146-8f1e9b2490ab admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:45.876368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1177/2388] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:16:45 2022] POST /identity/v3/users => generated 288 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:16:45.881265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.882822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:45.901996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0259d-b80e-4101-8b74-4905917a5871 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.907066 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:45.907860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.908326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.908795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.909600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:45.911627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:45.923785 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1212/2389] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:16:45.927952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f52738-5b8b-4d69-a146-8f1e9b2490ab admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.938957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f52738-5b8b-4d69-a146-8f1e9b2490ab admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.945800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b6c0d-d859-4dce-a6ae-89aaa86f17b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:45.946425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b6c0d-d859-4dce-a6ae-89aaa86f17b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.946770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b6c0d-d859-4dce-a6ae-89aaa86f17b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.947160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b6c0d-d859-4dce-a6ae-89aaa86f17b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.947691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b6c0d-d859-4dce-a6ae-89aaa86f17b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:45.949312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b6c0d-d859-4dce-a6ae-89aaa86f17b7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:45.956064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.958786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:45.966948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1178/2390] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:16:45.971395 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b6c0d-d859-4dce-a6ae-89aaa86f17b7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:45.985109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42095c16-d420-4d5c-9961-51c1cff42017 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.991715 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b6c0d-d859-4dce-a6ae-89aaa86f17b7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:45.992142 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:45.993111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:45.993569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:45.993938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] PATH_INFO: `/v3/projects/d11b2b1481db4b878ceee550360da8fe` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:45.995175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11b2b1481db4b878ceee550360da8fe)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:45.997287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:46.000622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cae3ca-00cf-4c92-80c8-de959a2d2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:46.001326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cae3ca-00cf-4c92-80c8-de959a2d2552 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:46.001925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cae3ca-00cf-4c92-80c8-de959a2d2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:46.002461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cae3ca-00cf-4c92-80c8-de959a2d2552 admin admin] PATH_INFO: `/v3/projects/faf0c6e98206493ea25654f44c9ae563/users/f12cfb692b4b4947bc67a3199aa33e73/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:46.024819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cae3ca-00cf-4c92-80c8-de959a2d2552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf0c6e98206493ea25654f44c9ae563, user_id=f12cfb692b4b4947bc67a3199aa33e73, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:46.028491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cae3ca-00cf-4c92-80c8-de959a2d2552 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:46.055285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1179/2391] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:16:45 2022] PUT /identity/v3/projects/faf0c6e98206493ea25654f44c9ae563/users/f12cfb692b4b4947bc67a3199aa33e73/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:46.057119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142504c-ac9c-4efb-a15c-abe205bce568 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:46.057401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142504c-ac9c-4efb-a15c-abe205bce568 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:46.057740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142504c-ac9c-4efb-a15c-abe205bce568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:46.070343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11b2b1481db4b878ceee550360da8fe', 'request_id': 'req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:46.078560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11b2b1481db4b878ceee550360da8fe', 'request_id': 'req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:46.085294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9142504c-ac9c-4efb-a15c-abe205bce568 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:46.086533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9142504c-ac9c-4efb-a15c-abe205bce568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:46.088699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11b2b1481db4b878ceee550360da8fe', 'request_id': 'req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:46.105275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] The token cache is being invalidate because project d11b2b1481db4b878ceee550360da8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:16:46.105982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:46.106907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:16:46.108704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1213/2392] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:16:45 2022] DELETE /identity/v3/projects/d11b2b1481db4b878ceee550360da8fe => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:46.201148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1180/2393] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:16:46.211426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:46.212937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:46.308520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84b01-d49b-4d46-8a99-70f3953ee4d1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:46.308749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:46.309487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:46.309771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:46.310044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:46.322328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:46.323833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:46.335574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1214/2394] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:16:51.775897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4334-e32d-4c42-9091-6f0bfef71715 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:51.775897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4334-e32d-4c42-9091-6f0bfef71715 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:51.776634 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f4334-e32d-4c42-9091-6f0bfef71715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:51.815325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-864f4334-e32d-4c42-9091-6f0bfef71715 None None] MFA Rules not processed for user `f12cfb692b4b4947bc67a3199aa33e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:51.816646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-864f4334-e32d-4c42-9091-6f0bfef71715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:51.877783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:51.920735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1181/2395] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:51.921829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:51.923545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:51.940659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa715c2-cdf8-411d-afc7-0c643c57d975 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:51.946388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4b9bd-ba4f-4756-b90d-1a3bcbcce193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:51.947012 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4b9bd-ba4f-4756-b90d-1a3bcbcce193 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:51.947349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4b9bd-ba4f-4756-b90d-1a3bcbcce193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:51.947631 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4b9bd-ba4f-4756-b90d-1a3bcbcce193 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:51.948391 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4b9bd-ba4f-4756-b90d-1a3bcbcce193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:51.950609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4b9bd-ba4f-4756-b90d-1a3bcbcce193 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:51.962069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:51.994819 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1215/2396] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:16:51 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:16:51.997000 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f4334-e32d-4c42-9091-6f0bfef71715 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:52.009733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f4334-e32d-4c42-9091-6f0bfef71715 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:52.071252 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b1a70-1804-406d-8ae4-fed112e64367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:52.071964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b1a70-1804-406d-8ae4-fed112e64367 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:52.072301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b1a70-1804-406d-8ae4-fed112e64367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:52.072662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b1a70-1804-406d-8ae4-fed112e64367 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:52.073288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b1a70-1804-406d-8ae4-fed112e64367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:52.075185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b1a70-1804-406d-8ae4-fed112e64367 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:52.101212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1182/2397] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:16:51 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:16:52.105959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4b9bd-ba4f-4756-b90d-1a3bcbcce193 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:52.119195 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4b9bd-ba4f-4756-b90d-1a3bcbcce193 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:52.125042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe861c-3d0c-4504-aa8f-eac4cc9741f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:52.126762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe861c-3d0c-4504-aa8f-eac4cc9741f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:52.127046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe861c-3d0c-4504-aa8f-eac4cc9741f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:52.127320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe861c-3d0c-4504-aa8f-eac4cc9741f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:52.127972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe861c-3d0c-4504-aa8f-eac4cc9741f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:52.129594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe861c-3d0c-4504-aa8f-eac4cc9741f2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:52.151017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1216/2398] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:16:52.154559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b1a70-1804-406d-8ae4-fed112e64367 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:52.166688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b1a70-1804-406d-8ae4-fed112e64367 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:52.172864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10e7a6-f826-4679-9038-56265d797a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:52.173946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10e7a6-f826-4679-9038-56265d797a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:52.174358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10e7a6-f826-4679-9038-56265d797a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:52.174693 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10e7a6-f826-4679-9038-56265d797a7d admin admin] PATH_INFO: `/v3/projects/c0a5d36de68946cd8d14aac3fdbdbd10/users/66ab32c245f04b5baab7426dfe1585d6/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:52.201808 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10e7a6-f826-4679-9038-56265d797a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a5d36de68946cd8d14aac3fdbdbd10, user_id=66ab32c245f04b5baab7426dfe1585d6, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:52.204747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10e7a6-f826-4679-9038-56265d797a7d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:52.223398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1183/2399] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:16:52 2022] PUT /identity/v3/projects/c0a5d36de68946cd8d14aac3fdbdbd10/users/66ab32c245f04b5baab7426dfe1585d6/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:16:52.227643 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe861c-3d0c-4504-aa8f-eac4cc9741f2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:52.240243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe861c-3d0c-4504-aa8f-eac4cc9741f2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:52.300961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71a9bc-f6d4-45bd-b1ef-304ed298485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:52.301230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71a9bc-f6d4-45bd-b1ef-304ed298485e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:52.301508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71a9bc-f6d4-45bd-b1ef-304ed298485e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:52.301780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71a9bc-f6d4-45bd-b1ef-304ed298485e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:52.303069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71a9bc-f6d4-45bd-b1ef-304ed298485e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:52.305039 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71a9bc-f6d4-45bd-b1ef-304ed298485e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:52.320917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10e7a6-f826-4679-9038-56265d797a7d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:52.332682 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1217/2400] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:16:52.337263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10e7a6-f826-4679-9038-56265d797a7d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:52.345199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:52.346010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:52.346358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:52.346702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] PATH_INFO: `/v3/projects/c0a5d36de68946cd8d14aac3fdbdbd10/users/66ab32c245f04b5baab7426dfe1585d6/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:52.348990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a5d36de68946cd8d14aac3fdbdbd10, user_id=66ab32c245f04b5baab7426dfe1585d6, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:52.352155 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:52.370118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1184/2401] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:16:52 2022] PUT /identity/v3/projects/c0a5d36de68946cd8d14aac3fdbdbd10/users/66ab32c245f04b5baab7426dfe1585d6/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:16:52.653317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac06d41-bc2c-4c4e-b2db-dbec8bd4e9b4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:52.653803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac06d41-bc2c-4c4e-b2db-dbec8bd4e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:52.654162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac06d41-bc2c-4c4e-b2db-dbec8bd4e9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:52.705445 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fac06d41-bc2c-4c4e-b2db-dbec8bd4e9b4 None None] MFA Rules not processed for user `871ff162be0b43c98b531e860f6a02e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:52.710182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fac06d41-bc2c-4c4e-b2db-dbec8bd4e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:52.812996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1218/2402] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:16:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:54.410843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:54.414607 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:54.453089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac06d41-bc2c-4c4e-b2db-dbec8bd4e9b4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:54.454805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fac06d41-bc2c-4c4e-b2db-dbec8bd4e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:54.537975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08e37-8bd1-45df-91d6-a28a6214ae7a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:54.553366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7857e-a913-416b-b72a-6333ee697f78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:54.554411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7857e-a913-416b-b72a-6333ee697f78 service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:54.554895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7857e-a913-416b-b72a-6333ee697f78 service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:54.555401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7857e-a913-416b-b72a-6333ee697f78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:54.558401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-67b7857e-a913-416b-b72a-6333ee697f78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:54.574425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7857e-a913-416b-b72a-6333ee697f78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:54.576707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7857e-a913-416b-b72a-6333ee697f78 service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:54.616104 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1185/2403] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:54.634605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac06d41-bc2c-4c4e-b2db-dbec8bd4e9b4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:54.662531 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc1a78-f495-40ab-bbf1-db482619215b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:54.663226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1a78-f495-40ab-bbf1-db482619215b service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:54.663511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1a78-f495-40ab-bbf1-db482619215b service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:54.663791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc1a78-f495-40ab-bbf1-db482619215b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:54.688355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc1a78-f495-40ab-bbf1-db482619215b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:54.691507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc1a78-f495-40ab-bbf1-db482619215b service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:54.719991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1219/2404] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:16:57.674420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade686d-f720-43a2-8d05-ceefcae1cfac None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:57.674932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade686d-f720-43a2-8d05-ceefcae1cfac None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:57.674932 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade686d-f720-43a2-8d05-ceefcae1cfac None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:57.676176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1186/2405] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:16:57.693585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8a56e-9994-40d9-8e19-7f418219bbad None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:57.694877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8a56e-9994-40d9-8e19-7f418219bbad None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:57.695532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8a56e-9994-40d9-8e19-7f418219bbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:57.716262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:57.750574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ddc8a56e-9994-40d9-8e19-7f418219bbad None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:57.752789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ddc8a56e-9994-40d9-8e19-7f418219bbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:57.930032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1220/2406] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:16:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:57.953276 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade686d-f720-43a2-8d05-ceefcae1cfac None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:57.955826 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9ade686d-f720-43a2-8d05-ceefcae1cfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:58.017633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:16:58.053785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade686d-f720-43a2-8d05-ceefcae1cfac None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:58.059140 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5a4e-f413-4ed9-95f0-81668e5de5fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a88817f3a4402290e9358826a63904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:58.060235 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5a4e-f413-4ed9-95f0-81668e5de5fc service glance] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:58.060766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5a4e-f413-4ed9-95f0-81668e5de5fc service glance] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:58.061414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5a4e-f413-4ed9-95f0-81668e5de5fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:58.078734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a5a4e-f413-4ed9-95f0-81668e5de5fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:58.080769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a5a4e-f413-4ed9-95f0-81668e5de5fc service glance] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:58.107448 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1187/2407] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:58.212503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31f389-08a5-4287-ac04-25061ed9b17b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:58.213031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31f389-08a5-4287-ac04-25061ed9b17b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:58.213818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31f389-08a5-4287-ac04-25061ed9b17b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:58.215630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1221/2408] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:16:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:16:58.220859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a93af-0fcb-46e4-873b-9d40340c00e7 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:58.221320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a93af-0fcb-46e4-873b-9d40340c00e7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:58.221879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a93af-0fcb-46e4-873b-9d40340c00e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:58.274603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bb2a93af-0fcb-46e4-873b-9d40340c00e7 None None] MFA Rules not processed for user `31183ac0fdd642e6a1a7578899aa9fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:58.276765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bb2a93af-0fcb-46e4-873b-9d40340c00e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:58.368285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1188/2409] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:16:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:16:58.380419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31f389-08a5-4287-ac04-25061ed9b17b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:58.383011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fe31f389-08a5-4287-ac04-25061ed9b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:58.415471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31f389-08a5-4287-ac04-25061ed9b17b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:58.420134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667ccb-c197-47d6-9200-8e47023dd7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:58.420806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667ccb-c197-47d6-9200-8e47023dd7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:58.421082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667ccb-c197-47d6-9200-8e47023dd7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:58.421351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667ccb-c197-47d6-9200-8e47023dd7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:58.433629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd667ccb-c197-47d6-9200-8e47023dd7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:58.435530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd667ccb-c197-47d6-9200-8e47023dd7e4 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:58.448505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1222/2410] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:16:58.603940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb52fd-f84d-41df-9d91-9e5a45f8eb84 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:58.604143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb52fd-f84d-41df-9d91-9e5a45f8eb84 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:58.604573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb52fd-f84d-41df-9d91-9e5a45f8eb84 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:58.606548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1189/2411] 10.222.0.70 () {54 vars in 921 bytes} [Tue Aug 9 19:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 19:16:58.612863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99c28-7497-4c69-a584-f4e7dd206af1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:58.613471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99c28-7497-4c69-a584-f4e7dd206af1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:58.613953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99c28-7497-4c69-a584-f4e7dd206af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:58.643951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e3c99c28-7497-4c69-a584-f4e7dd206af1 None None] MFA Rules not processed for user `48410d57fe0d489cbb07cfca58ca7b30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:16:58.647472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e3c99c28-7497-4c69-a584-f4e7dd206af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:16:58.665525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1223/2412] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Aug 9 19:16:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:16:58.679562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb52fd-f84d-41df-9d91-9e5a45f8eb84 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:16:58.691256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb52fd-f84d-41df-9d91-9e5a45f8eb84 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:16:58.696266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:16:58.697220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:16:58.697491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:16:58.697921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:16:58.719957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:16:58.721583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:16:58.735067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1190/2413] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:16:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:17:00.672042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe59a-d924-4c2d-8ac4-5b6882af755e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:17:00.672594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe59a-d924-4c2d-8ac4-5b6882af755e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:17:00.672822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe59a-d924-4c2d-8ac4-5b6882af755e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:17:00.706489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f30fe59a-d924-4c2d-8ac4-5b6882af755e None None] MFA Rules not processed for user `66ab32c245f04b5baab7426dfe1585d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:17:00.707768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f30fe59a-d924-4c2d-8ac4-5b6882af755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:17:00.800714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1224/2414] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:17:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:17:00.823075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:17:00.825440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:17:00.853627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c334df-f653-479c-9cf4-e7c064bec497 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:17:00.859612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:17:00.860571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:17:00.861011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:17:00.861429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:17:00.925197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:17:00.926706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:17:00.937266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1191/2415] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:17:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:17:02.658589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f2ef-a030-4d64-8b89-b50d9efe0f24 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:17:02.659072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f2ef-a030-4d64-8b89-b50d9efe0f24 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:17:02.659226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f2ef-a030-4d64-8b89-b50d9efe0f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:17:02.693382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5080f2ef-a030-4d64-8b89-b50d9efe0f24 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:17:02.697016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5080f2ef-a030-4d64-8b89-b50d9efe0f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:17:02.800135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1225/2416] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:17:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:17:02.813196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:17:02.815095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:17:02.921733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6518eb-409b-4574-804e-c39bdf7728da service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:17:02.927990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d2c14-ce23-4c7d-8903-0328ec444a0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:17:02.928916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d2c14-ce23-4c7d-8903-0328ec444a0d service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:17:02.929349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d2c14-ce23-4c7d-8903-0328ec444a0d service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:17:02.929943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d2c14-ce23-4c7d-8903-0328ec444a0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:17:02.939725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d2c14-ce23-4c7d-8903-0328ec444a0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:17:02.941648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d2c14-ce23-4c7d-8903-0328ec444a0d service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:17:02.947355 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:17:02.951986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1192/2417] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:17:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:17:12.789924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:17:19.354408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1dd55-5e2b-494f-bcf0-f21eb1849eb2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:17:19.354992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1dd55-5e2b-494f-bcf0-f21eb1849eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:17:19.354992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1dd55-5e2b-494f-bcf0-f21eb1849eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:17:19.373461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-94f1dd55-5e2b-494f-bcf0-f21eb1849eb2 None None] MFA Rules not processed for user `f12cfb692b4b4947bc67a3199aa33e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:17:19.374642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-94f1dd55-5e2b-494f-bcf0-f21eb1849eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:17:19.381965 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1226/2418] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:17:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:17:26.563710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc94c95-c5ae-47f7-b349-1b7c6e73dd92 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:17:26.564268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc94c95-c5ae-47f7-b349-1b7c6e73dd92 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:17:26.564268 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc94c95-c5ae-47f7-b349-1b7c6e73dd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:17:26.585835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-efc94c95-c5ae-47f7-b349-1b7c6e73dd92 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:17:26.587287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-efc94c95-c5ae-47f7-b349-1b7c6e73dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:17:26.594464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1193/2419] 10.222.0.70 () {58 vars in 967 bytes} [Tue Aug 9 19:17:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:17:26.617105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1dd55-5e2b-494f-bcf0-f21eb1849eb2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:17:26.629372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1dd55-5e2b-494f-bcf0-f21eb1849eb2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:17:26.635075 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e33012-dfa0-4c0b-b914-bebbaaa20f7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:17:26.635678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e33012-dfa0-4c0b-b914-bebbaaa20f7f service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:17:26.635957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e33012-dfa0-4c0b-b914-bebbaaa20f7f service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:17:26.636227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e33012-dfa0-4c0b-b914-bebbaaa20f7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:17:26.644804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e33012-dfa0-4c0b-b914-bebbaaa20f7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:17:26.646376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e33012-dfa0-4c0b-b914-bebbaaa20f7f service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:17:26.656705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1227/2420] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:17:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:17:36.579517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:17:36.653236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=9 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:17:43.934965 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc94c95-c5ae-47f7-b349-1b7c6e73dd92 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:17:43.943523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc94c95-c5ae-47f7-b349-1b7c6e73dd92 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:17:43.947422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:17:43.948026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:17:43.948293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:17:43.948599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:17:43.957342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:17:43.958752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:17:43.968385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1194/2421] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:17:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:17:53.964260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:06.785887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a5a3-7fec-4ac6-bc6a-a04b7642d52d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:06.786409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a5a3-7fec-4ac6-bc6a-a04b7642d52d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:06.786409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a5a3-7fec-4ac6-bc6a-a04b7642d52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:06.806183 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bac4a5a3-7fec-4ac6-bc6a-a04b7642d52d None None] MFA Rules not processed for user `8755b16a18734175b691f891351d5116`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:06.807365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bac4a5a3-7fec-4ac6-bc6a-a04b7642d52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:06.861172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1228/2422] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:18:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:12.639360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:12.641458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:12.709215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9d65-1ed9-47ee-bad9-5763f1a1419d service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:12.714709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:12.715328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:12.715609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:12.715885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] PATH_INFO: `/v3/users/871ff162be0b43c98b531e860f6a02e5` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:12.717038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=871ff162be0b43c98b531e860f6a02e5)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:12.718956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:12.765137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871ff162be0b43c98b531e860f6a02e5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:12.774153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871ff162be0b43c98b531e860f6a02e5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:12.781208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871ff162be0b43c98b531e860f6a02e5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:12.781845 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871ff162be0b43c98b531e860f6a02e5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:12.782456 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-99fcb64a-c101-4b0f-8e6b-d21df1d00906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871ff162be0b43c98b531e860f6a02e5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:12.800404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1195/2423] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:18:12 2022] DELETE /identity/v3/users/871ff162be0b43c98b531e860f6a02e5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:18:12.973135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4a5a3-7fec-4ac6-bc6a-a04b7642d52d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:12.984454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4a5a3-7fec-4ac6-bc6a-a04b7642d52d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:13.045080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:13.045716 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.045995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.046269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] PATH_INFO: `/v3/projects/37a9557289ea4fe9903f6cf421d937d1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.047653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a9557289ea4fe9903f6cf421d937d1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:13.049610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:13.096029 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a9557289ea4fe9903f6cf421d937d1', 'request_id': 'req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:13.102099 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a9557289ea4fe9903f6cf421d937d1', 'request_id': 'req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:13.109298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a9557289ea4fe9903f6cf421d937d1', 'request_id': 'req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:13.121422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] The token cache is being invalidate because project 37a9557289ea4fe9903f6cf421d937d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:18:13.121720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:13.122367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e0c70b2-6ffc-4f2b-ab6a-6656bec45546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:18:13.123976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1229/2424] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:18:12 2022] DELETE /identity/v3/projects/37a9557289ea4fe9903f6cf421d937d1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:18:13.129270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f941b-141a-4e40-a7e5-4ffc35f924d9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.129556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f941b-141a-4e40-a7e5-4ffc35f924d9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.129834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f941b-141a-4e40-a7e5-4ffc35f924d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.149374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9f9f941b-141a-4e40-a7e5-4ffc35f924d9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:13.150495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9f9f941b-141a-4e40-a7e5-4ffc35f924d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:13.209363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1196/2425] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:13.217929 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d5cbf-c5b6-4225-83fc-630bc68f512c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.218216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d5cbf-c5b6-4225-83fc-630bc68f512c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.218497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d5cbf-c5b6-4225-83fc-630bc68f512c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.240079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ce6d5cbf-c5b6-4225-83fc-630bc68f512c None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:13.242254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce6d5cbf-c5b6-4225-83fc-630bc68f512c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:13.250144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1230/2426] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:13.254351 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f941b-141a-4e40-a7e5-4ffc35f924d9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:13.264034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f941b-141a-4e40-a7e5-4ffc35f924d9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:13.268604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:13.269196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.269471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.269740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.270376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:18:13.270673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:13.272497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:13.283974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1197/2427] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:18:13.288829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d5cbf-c5b6-4225-83fc-630bc68f512c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:13.298239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d5cbf-c5b6-4225-83fc-630bc68f512c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:13.302823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b728e3c-e1fb-45d2-b22e-fca0703c7c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:13.304027 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728e3c-e1fb-45d2-b22e-fca0703c7c65 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.304300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728e3c-e1fb-45d2-b22e-fca0703c7c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.304605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728e3c-e1fb-45d2-b22e-fca0703c7c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.305222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b728e3c-e1fb-45d2-b22e-fca0703c7c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:13.307108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b728e3c-e1fb-45d2-b22e-fca0703c7c65 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:13.334578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1231/2428] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:18:13 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:18:13.339047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:13.350956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcd32e-98b1-43cd-a619-bc4c056de648 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:13.354853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:13.408948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21387f9-a12d-4404-9b6b-9c29c67711e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:13.409555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21387f9-a12d-4404-9b6b-9c29c67711e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.409828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21387f9-a12d-4404-9b6b-9c29c67711e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.410102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21387f9-a12d-4404-9b6b-9c29c67711e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.410724 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21387f9-a12d-4404-9b6b-9c29c67711e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:13.413238 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21387f9-a12d-4404-9b6b-9c29c67711e2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:13.434587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1198/2429] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:18:13 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:18:13.438861 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b728e3c-e1fb-45d2-b22e-fca0703c7c65 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:13.449119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b728e3c-e1fb-45d2-b22e-fca0703c7c65 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:13.453644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff18b-20a0-4475-90cd-205415a22cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:13.454236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff18b-20a0-4475-90cd-205415a22cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.454512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff18b-20a0-4475-90cd-205415a22cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.454784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff18b-20a0-4475-90cd-205415a22cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.455416 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cff18b-20a0-4475-90cd-205415a22cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:13.457038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cff18b-20a0-4475-90cd-205415a22cf9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:13.458262 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:13.470346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1232/2430] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:18:13.474281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21387f9-a12d-4404-9b6b-9c29c67711e2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:13.485421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21387f9-a12d-4404-9b6b-9c29c67711e2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:13.490048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab80c50-bb70-48dd-a7ad-1eb425ef9611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:13.490664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab80c50-bb70-48dd-a7ad-1eb425ef9611 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.491055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab80c50-bb70-48dd-a7ad-1eb425ef9611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.491338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab80c50-bb70-48dd-a7ad-1eb425ef9611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.491985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab80c50-bb70-48dd-a7ad-1eb425ef9611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:13.493623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab80c50-bb70-48dd-a7ad-1eb425ef9611 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:13.504083 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1199/2431] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:18:13.508220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff18b-20a0-4475-90cd-205415a22cf9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:13.518322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff18b-20a0-4475-90cd-205415a22cf9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:13.523662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:13.524303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.524621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.524927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] PATH_INFO: `/v3/projects/63c5b147e9d0494381a36bd407f110db/users/cbec728603924ccabfffc6cc8d18c5c9/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.541291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c5b147e9d0494381a36bd407f110db, user_id=cbec728603924ccabfffc6cc8d18c5c9, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:13.543942 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:13.557082 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1233/2432] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:18:13 2022] PUT /identity/v3/projects/63c5b147e9d0494381a36bd407f110db/users/cbec728603924ccabfffc6cc8d18c5c9/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:18:13.562136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d293d-1fdc-4397-88e6-126b013356db None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.562488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d293d-1fdc-4397-88e6-126b013356db None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.562764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d293d-1fdc-4397-88e6-126b013356db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.582779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-377d293d-1fdc-4397-88e6-126b013356db None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:13.583980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-377d293d-1fdc-4397-88e6-126b013356db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:13.641928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1200/2433] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:18:13.651894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:13.653580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:13.708989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93be5e6-8ce5-42f2-b8b6-6840ccbe5940 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:13.713056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c1222-f4c3-4818-9a41-4ec8048c845b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:13.713647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c1222-f4c3-4818-9a41-4ec8048c845b service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:13.713917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c1222-f4c3-4818-9a41-4ec8048c845b service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:13.714181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c1222-f4c3-4818-9a41-4ec8048c845b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:13.722961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c1222-f4c3-4818-9a41-4ec8048c845b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:13.724383 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c1222-f4c3-4818-9a41-4ec8048c845b service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:13.734157 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1234/2434] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:18:18.204008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80647ca8-21da-411b-8015-958a26e6f74f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:18.204513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80647ca8-21da-411b-8015-958a26e6f74f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:18.204629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80647ca8-21da-411b-8015-958a26e6f74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:18.238177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-80647ca8-21da-411b-8015-958a26e6f74f None None] MFA Rules not processed for user `cbec728603924ccabfffc6cc8d18c5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:18.239406 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-80647ca8-21da-411b-8015-958a26e6f74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:18.310780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1201/2435] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:18:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:18.315497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c1222-f4c3-4818-9a41-4ec8048c845b service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:18.325922 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c1222-f4c3-4818-9a41-4ec8048c845b service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:18.330982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6283e-e99a-42e7-9860-a2fef7491f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:18.331886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6283e-e99a-42e7-9860-a2fef7491f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:18.332022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6283e-e99a-42e7-9860-a2fef7491f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:18.332290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6283e-e99a-42e7-9860-a2fef7491f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:18.332985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6283e-e99a-42e7-9860-a2fef7491f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:18.335607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6283e-e99a-42e7-9860-a2fef7491f74 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:18.367495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1235/2436] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:18:18 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:18:18.371712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80647ca8-21da-411b-8015-958a26e6f74f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:18.383585 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80647ca8-21da-411b-8015-958a26e6f74f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:18.399078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:18.453956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e705ddc-7fa6-4520-b8bc-208642729df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:18.454718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e705ddc-7fa6-4520-b8bc-208642729df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:18.455061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e705ddc-7fa6-4520-b8bc-208642729df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:18.455429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e705ddc-7fa6-4520-b8bc-208642729df1 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:18.456151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e705ddc-7fa6-4520-b8bc-208642729df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:18.458234 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e705ddc-7fa6-4520-b8bc-208642729df1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:18.483672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1202/2437] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:18:18 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:18:18.487372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6283e-e99a-42e7-9860-a2fef7491f74 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:18.499322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6283e-e99a-42e7-9860-a2fef7491f74 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:18.505714 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3344d2-88b0-4167-9d11-bc53cf551b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:18.506887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3344d2-88b0-4167-9d11-bc53cf551b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:18.507159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3344d2-88b0-4167-9d11-bc53cf551b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:18.507603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3344d2-88b0-4167-9d11-bc53cf551b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:18.508081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3344d2-88b0-4167-9d11-bc53cf551b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:18.509739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3344d2-88b0-4167-9d11-bc53cf551b20 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:18.525036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1236/2438] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:18:18.526320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e705ddc-7fa6-4520-b8bc-208642729df1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:18.537068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e705ddc-7fa6-4520-b8bc-208642729df1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:18.541787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92195fd4-dbbb-4c65-b69b-cff4ee2f33b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:18.542567 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92195fd4-dbbb-4c65-b69b-cff4ee2f33b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:18.542846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92195fd4-dbbb-4c65-b69b-cff4ee2f33b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:18.543115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92195fd4-dbbb-4c65-b69b-cff4ee2f33b9 admin admin] PATH_INFO: `/v3/projects/5e71118ace3e4342b88920494187374a/users/2c0979f7d3c04ead9cd8b12a1b07cb33/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:18.558780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92195fd4-dbbb-4c65-b69b-cff4ee2f33b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e71118ace3e4342b88920494187374a, user_id=2c0979f7d3c04ead9cd8b12a1b07cb33, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:18.562707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92195fd4-dbbb-4c65-b69b-cff4ee2f33b9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:18.578341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1203/2439] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:18:18 2022] PUT /identity/v3/projects/5e71118ace3e4342b88920494187374a/users/2c0979f7d3c04ead9cd8b12a1b07cb33/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:18:18.582209 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3344d2-88b0-4167-9d11-bc53cf551b20 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:18.593344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3344d2-88b0-4167-9d11-bc53cf551b20 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:18.652051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2d808-ac36-4f73-be8c-f86684c8b739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:18.652703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2d808-ac36-4f73-be8c-f86684c8b739 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:18.652979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2d808-ac36-4f73-be8c-f86684c8b739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:18.653247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2d808-ac36-4f73-be8c-f86684c8b739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:18.653881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2d808-ac36-4f73-be8c-f86684c8b739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:18.655462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2d808-ac36-4f73-be8c-f86684c8b739 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:18.665466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1237/2440] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:18:18.669685 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92195fd4-dbbb-4c65-b69b-cff4ee2f33b9 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:18.681409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92195fd4-dbbb-4c65-b69b-cff4ee2f33b9 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:18.686010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ef9f3-8934-460b-bbb1-f863bd357845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:18.686659 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ef9f3-8934-460b-bbb1-f863bd357845 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:18.686934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ef9f3-8934-460b-bbb1-f863bd357845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:18.687261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ef9f3-8934-460b-bbb1-f863bd357845 admin admin] PATH_INFO: `/v3/projects/5e71118ace3e4342b88920494187374a/users/2c0979f7d3c04ead9cd8b12a1b07cb33/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:18.689409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ef9f3-8934-460b-bbb1-f863bd357845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e71118ace3e4342b88920494187374a, user_id=2c0979f7d3c04ead9cd8b12a1b07cb33, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:18.692026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ef9f3-8934-460b-bbb1-f863bd357845 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:18.704166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1204/2441] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:18:18 2022] PUT /identity/v3/projects/5e71118ace3e4342b88920494187374a/users/2c0979f7d3c04ead9cd8b12a1b07cb33/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:18:25.023314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd722f-09ed-43e1-8fa0-9ef276e84165 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:25.023724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd722f-09ed-43e1-8fa0-9ef276e84165 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:25.025497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd722f-09ed-43e1-8fa0-9ef276e84165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:25.044985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289f0b-2473-4d3b-b4fe-c7d5ce3df59e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:25.045444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289f0b-2473-4d3b-b4fe-c7d5ce3df59e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:25.045537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289f0b-2473-4d3b-b4fe-c7d5ce3df59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:25.055323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b8cd722f-09ed-43e1-8fa0-9ef276e84165 None None] MFA Rules not processed for user `e9ad80a549b14b25ac71cd96832a74e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:25.057017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b8cd722f-09ed-43e1-8fa0-9ef276e84165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:25.100865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:25.112037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d7289f0b-2473-4d3b-b4fe-c7d5ce3df59e None None] MFA Rules not processed for user `2c0979f7d3c04ead9cd8b12a1b07cb33`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:25.113719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d7289f0b-2473-4d3b-b4fe-c7d5ce3df59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:25.143061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1238/2442] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:18:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:25.206896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1205/2443] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:18:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:25.222405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd722f-09ed-43e1-8fa0-9ef276e84165 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:25.227736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b8cd722f-09ed-43e1-8fa0-9ef276e84165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:25.313432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd722f-09ed-43e1-8fa0-9ef276e84165 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:25.319019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a88817f3a4402290e9358826a63904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:25.319019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:25.319019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:25.319544 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:25.418863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:25.422070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:25.433836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1239/2444] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:18:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:18:25.632383 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4347a7-5373-4708-bc93-90a5e664f7bf None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:25.633312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4347a7-5373-4708-bc93-90a5e664f7bf None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:25.633806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4347a7-5373-4708-bc93-90a5e664f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:25.699391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-fd4347a7-5373-4708-bc93-90a5e664f7bf None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:25.700949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fd4347a7-5373-4708-bc93-90a5e664f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:25.797380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:25.808859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1206/2445] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:18:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:25.875839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:25.877453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:25.978432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116299f2-77b3-4f7c-92e5-d4730e8928f8 service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:25.983737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87656d4a-cd5c-4667-b30d-089b062de65b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:25.984489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656d4a-cd5c-4667-b30d-089b062de65b service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:25.984848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656d4a-cd5c-4667-b30d-089b062de65b service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:25.985189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656d4a-cd5c-4667-b30d-089b062de65b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:25.994661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87656d4a-cd5c-4667-b30d-089b062de65b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:25.996369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87656d4a-cd5c-4667-b30d-089b062de65b service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:26.006686 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1240/2446] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:18:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:18:35.811896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:36.002318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:36.081598 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b9b46-2de6-4445-9f0c-183a57e48a26 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:36.082091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b9b46-2de6-4445-9f0c-183a57e48a26 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:36.082577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b9b46-2de6-4445-9f0c-183a57e48a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:36.109545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-772b9b46-2de6-4445-9f0c-183a57e48a26 None None] MFA Rules not processed for user `cbec728603924ccabfffc6cc8d18c5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:36.110749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-772b9b46-2de6-4445-9f0c-183a57e48a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:36.118563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1207/2447] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:18:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:38.249906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d64ce-b2d4-41f0-af9a-10a6fb0a776f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:38.250398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d64ce-b2d4-41f0-af9a-10a6fb0a776f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:38.250496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d64ce-b2d4-41f0-af9a-10a6fb0a776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:38.271733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bb6d64ce-b2d4-41f0-af9a-10a6fb0a776f None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:18:38.273202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bb6d64ce-b2d4-41f0-af9a-10a6fb0a776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:38.281784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1241/2448] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:18:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:18:38.297260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b9b46-2de6-4445-9f0c-183a57e48a26 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:38.310646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b9b46-2de6-4445-9f0c-183a57e48a26 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:38.316182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:38.316837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:38.317284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:38.317741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:38.327695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:38.329327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:38.338605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1208/2449] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:18:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:18:48.265300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:48.334873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=9 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:18:49.948090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d64ce-b2d4-41f0-af9a-10a6fb0a776f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:49.949617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bb6d64ce-b2d4-41f0-af9a-10a6fb0a776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:50.016657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d64ce-b2d4-41f0-af9a-10a6fb0a776f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:50.020483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:50.021789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:50.023778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:50.023778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:50.024256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:50.101198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:50.102639 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:50.113758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1242/2450] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:18:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:18:53.952163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:18:53.953582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:54.010980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02b96c-a242-4fca-9c3c-05041943a6ec service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:18:54.014691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:18:54.015328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:18:54.015603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:18:54.015877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:18:54.017626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:18:54.031003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:18:54.032413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:18:54.046981 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1209/2451] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:18:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:00.108794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=54 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:01.724421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:01.733314 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3342c-7eb8-4777-a62c-b99d9f44b9ba service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:01.737002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:01.737596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:01.737875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:01.738150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:01.739894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:01.799399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:01.800869 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:01.810595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1243/2452] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:04.040852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=26 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:06.551280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:06.560258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf439f-8a46-48ac-94bb-f5482fd8e60e service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:06.563908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:06.564606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:06.564887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:06.565161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:06.574922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:06.576333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:06.586542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1210/2453] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:11.750110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:11.760235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805131c0-64c2-4c4c-8134-cfecabf2a1df service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:11.763843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:11.764431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:11.764791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:11.765274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:11.767996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:11.834340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:11.836986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:11.847193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1244/2454] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:14.923167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:14.924718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:14.949376 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af318515-e871-45bf-9a3c-a73fa65e2686 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:14.955629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:14.956833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:14.957011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:14.957506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] PATH_INFO: `/v3/users/8755b16a18734175b691f891351d5116` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:14.963537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8755b16a18734175b691f891351d5116)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:14.965934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:15.017590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8755b16a18734175b691f891351d5116'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.026494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8755b16a18734175b691f891351d5116'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.033994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8755b16a18734175b691f891351d5116'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.034660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8755b16a18734175b691f891351d5116'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.035259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6762f774-4673-4042-ad8a-201642f76285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8755b16a18734175b691f891351d5116'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.057394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1211/2455] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:19:14 2022] DELETE /identity/v3/users/8755b16a18734175b691f891351d5116 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:19:15.233325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.246171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef808c-83e8-4d8d-9e3f-61c01bdffd35 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:15.304276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:15.305002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.305281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.305555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] PATH_INFO: `/v3/projects/375ce88986dc47b08134e076562eee05` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.306692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375ce88986dc47b08134e076562eee05)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:15.308647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:15.346519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:15.357328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375ce88986dc47b08134e076562eee05', 'request_id': 'req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.362804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375ce88986dc47b08134e076562eee05', 'request_id': 'req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.370188 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375ce88986dc47b08134e076562eee05', 'request_id': 'req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.381235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] The token cache is being invalidate because project 375ce88986dc47b08134e076562eee05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:19:15.381540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.382851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50c0cfff-c501-4dfc-bd72-41e7e4f5c566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:19:15.384569 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1245/2456] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:19:15 2022] DELETE /identity/v3/projects/375ce88986dc47b08134e076562eee05 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:19:15.391019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37669511-1749-4fb8-ae95-0575b261acf6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.391305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37669511-1749-4fb8-ae95-0575b261acf6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.391581 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37669511-1749-4fb8-ae95-0575b261acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.411518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-37669511-1749-4fb8-ae95-0575b261acf6 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:19:15.412735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-37669511-1749-4fb8-ae95-0575b261acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:15.472010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1212/2457] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:19:15.484094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3dc6a-fefc-447e-bc1c-3023b9c2e11a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.484549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3dc6a-fefc-447e-bc1c-3023b9c2e11a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.484838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3dc6a-fefc-447e-bc1c-3023b9c2e11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.505568 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e0c3dc6a-fefc-447e-bc1c-3023b9c2e11a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:19:15.506782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e0c3dc6a-fefc-447e-bc1c-3023b9c2e11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:15.515420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1246/2458] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:19:15.519805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37669511-1749-4fb8-ae95-0575b261acf6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.530114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37669511-1749-4fb8-ae95-0575b261acf6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:15.534702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:15.535296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.535572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.535841 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.536519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:19:15.536825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:15.538537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:15.550194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1213/2459] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:19:15.554382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3dc6a-fefc-447e-bc1c-3023b9c2e11a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.563709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3dc6a-fefc-447e-bc1c-3023b9c2e11a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:15.568350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cd18-7bb8-4d35-b569-5630fc866fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:15.568988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cd18-7bb8-4d35-b569-5630fc866fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.569274 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cd18-7bb8-4d35-b569-5630fc866fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.570061 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cd18-7bb8-4d35-b569-5630fc866fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.570688 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4cd18-7bb8-4d35-b569-5630fc866fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:15.572611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4cd18-7bb8-4d35-b569-5630fc866fd2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:15.602955 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1247/2460] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:19:15 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:19:15.604878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.614700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3d19-34a8-43b6-b522-0552ff556a22 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:15.658509 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:15.673427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7434c-81c2-4531-a873-24bed5303b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:15.674071 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7434c-81c2-4531-a873-24bed5303b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.674335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7434c-81c2-4531-a873-24bed5303b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.674606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7434c-81c2-4531-a873-24bed5303b89 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.675228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7434c-81c2-4531-a873-24bed5303b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:15.677228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7434c-81c2-4531-a873-24bed5303b89 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:15.698464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1214/2461] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:19:15 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:19:15.702684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cd18-7bb8-4d35-b569-5630fc866fd2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.713120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cd18-7bb8-4d35-b569-5630fc866fd2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:15.718469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcfa07-50c1-4157-b4f7-cde410b89b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:15.719071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfa07-50c1-4157-b4f7-cde410b89b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.719351 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfa07-50c1-4157-b4f7-cde410b89b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.719624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfa07-50c1-4157-b4f7-cde410b89b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.720270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcfa07-50c1-4157-b4f7-cde410b89b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:15.721980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcfa07-50c1-4157-b4f7-cde410b89b86 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:15.732343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1248/2462] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:19:15.736243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7434c-81c2-4531-a873-24bed5303b89 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.746643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7434c-81c2-4531-a873-24bed5303b89 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:15.751297 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a3f24-bfcc-43af-9315-4289d6e8abe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:15.751912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a3f24-bfcc-43af-9315-4289d6e8abe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.752187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a3f24-bfcc-43af-9315-4289d6e8abe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.752517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a3f24-bfcc-43af-9315-4289d6e8abe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.753159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a3f24-bfcc-43af-9315-4289d6e8abe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:15.754731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a3f24-bfcc-43af-9315-4289d6e8abe9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:15.765764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1215/2463] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:19:15.769827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcfa07-50c1-4157-b4f7-cde410b89b86 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.779912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcfa07-50c1-4157-b4f7-cde410b89b86 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:15.784645 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:15.785281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.785560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.785833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] PATH_INFO: `/v3/projects/46afad886ad34608b533f4ee80a18ba5/users/4f7899ab15b347b7be75e3433f0d5450/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.800560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46afad886ad34608b533f4ee80a18ba5, user_id=4f7899ab15b347b7be75e3433f0d5450, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:15.803203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:15.816675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1249/2464] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:19:15 2022] PUT /identity/v3/projects/46afad886ad34608b533f4ee80a18ba5/users/4f7899ab15b347b7be75e3433f0d5450/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:19:15.822476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe65b0-f5ea-4004-ae03-a464fc63350a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.822758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe65b0-f5ea-4004-ae03-a464fc63350a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.823037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe65b0-f5ea-4004-ae03-a464fc63350a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.841341 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4cbe65b0-f5ea-4004-ae03-a464fc63350a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:19:15.842616 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4cbe65b0-f5ea-4004-ae03-a464fc63350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:15.906446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1216/2465] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:19:15.916780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.918473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:15.974973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152ed07-92b3-479f-a38d-b63fbbcedcd6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:15.978988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ce6aa-1b4c-4c5e-9127-a78dadd970b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:15.979584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce6aa-1b4c-4c5e-9127-a78dadd970b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:15.979858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce6aa-1b4c-4c5e-9127-a78dadd970b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:15.980128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce6aa-1b4c-4c5e-9127-a78dadd970b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:15.982611 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe65b0-f5ea-4004-ae03-a464fc63350a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:15.983994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4cbe65b0-f5ea-4004-ae03-a464fc63350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:15.989747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ce6aa-1b4c-4c5e-9127-a78dadd970b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:15.991176 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ce6aa-1b4c-4c5e-9127-a78dadd970b5 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:16.001487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1250/2466] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:19:16.059561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe65b0-f5ea-4004-ae03-a464fc63350a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:16.065156 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:16.065798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:16.066091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:16.066310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:16.068315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:16.130925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:16.133163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:16.142719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1217/2467] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:19:20.278119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227a99b-bd81-4be1-9782-05adac57e8b1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:20.278675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227a99b-bd81-4be1-9782-05adac57e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:20.278958 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227a99b-bd81-4be1-9782-05adac57e8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:20.325148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4227a99b-bd81-4be1-9782-05adac57e8b1 None None] MFA Rules not processed for user `4f7899ab15b347b7be75e3433f0d5450`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:19:20.326697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4227a99b-bd81-4be1-9782-05adac57e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:20.399008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:20.425855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1251/2468] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:19:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:19:20.631568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:20.638237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:20.648829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407bc7c-babc-40bf-b6d2-75e28197eff5 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:20.654871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e86c1-5c26-425d-b2c9-b6157dc00e95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:20.655987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e86c1-5c26-425d-b2c9-b6157dc00e95 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:20.656675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e86c1-5c26-425d-b2c9-b6157dc00e95 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:20.657758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e86c1-5c26-425d-b2c9-b6157dc00e95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:20.670133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e86c1-5c26-425d-b2c9-b6157dc00e95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:20.672075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e86c1-5c26-425d-b2c9-b6157dc00e95 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:20.683332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1218/2469] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:21.455978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227a99b-bd81-4be1-9782-05adac57e8b1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:21.472993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227a99b-bd81-4be1-9782-05adac57e8b1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:21.479108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:21.479749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:21.480023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:21.480293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:21.482580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:21.513487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:21.518152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:21.543384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1252/2470] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:22.332766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef191b-fad7-4105-8e2b-80772f5baa3e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:22.333347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef191b-fad7-4105-8e2b-80772f5baa3e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:22.333442 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef191b-fad7-4105-8e2b-80772f5baa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:22.355783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-61ef191b-fad7-4105-8e2b-80772f5baa3e None None] MFA Rules not processed for user `4f7899ab15b347b7be75e3433f0d5450`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:19:22.357146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-61ef191b-fad7-4105-8e2b-80772f5baa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:22.371198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1219/2471] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:19:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:19:22.712198 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:22.722344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee795cd-0208-4782-9d3f-49c8baa917d6 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:22.725974 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:22.726571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:22.726847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:22.727119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:22.728887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:22.739783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:22.742147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:22.753481 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1253/2472] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:23.144591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef191b-fad7-4105-8e2b-80772f5baa3e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:23.146160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-61ef191b-fad7-4105-8e2b-80772f5baa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:23.254004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef191b-fad7-4105-8e2b-80772f5baa3e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:23.267545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23951781-ca9b-4f16-bb19-b5c60003783c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:23.269495 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951781-ca9b-4f16-bb19-b5c60003783c service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:23.270307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951781-ca9b-4f16-bb19-b5c60003783c service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:23.270595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23951781-ca9b-4f16-bb19-b5c60003783c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:23.279387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-23951781-ca9b-4f16-bb19-b5c60003783c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:23.305044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23951781-ca9b-4f16-bb19-b5c60003783c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:23.306237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23951781-ca9b-4f16-bb19-b5c60003783c service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:23.332890 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1220/2473] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:25.626959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:25.640878 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d299e-f363-475d-9444-a5b83fee1029 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:25.645336 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:25.646016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:25.646303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:25.646651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:25.648957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:25.661315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:25.663497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:25.673771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1254/2474] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:25.872127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9302b-cc6b-4242-a720-43ba2b3c827e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:25.873029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9302b-cc6b-4242-a720-43ba2b3c827e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:25.873361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9302b-cc6b-4242-a720-43ba2b3c827e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:25.893880 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c9b9302b-cc6b-4242-a720-43ba2b3c827e None None] MFA Rules not processed for user `05147fb1762748ee9866ccc5bff4d514`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:19:25.895131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c9b9302b-cc6b-4242-a720-43ba2b3c827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:25.969484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1221/2475] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:19:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:19:26.319531 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:26.320960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:26.331286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d8be2-4849-4d40-9d21-cdaa60a9d870 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:26.335914 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c69b64-71f2-45d8-8376-f89a22b6902f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:26.336557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69b64-71f2-45d8-8376-f89a22b6902f service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:26.336835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69b64-71f2-45d8-8376-f89a22b6902f service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:26.337100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69b64-71f2-45d8-8376-f89a22b6902f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:26.338821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-53c69b64-71f2-45d8-8376-f89a22b6902f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:26.349235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c69b64-71f2-45d8-8376-f89a22b6902f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:26.351388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c69b64-71f2-45d8-8376-f89a22b6902f service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:26.361407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1255/2476] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:29.937250 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9302b-cc6b-4242-a720-43ba2b3c827e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:29.951597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9302b-cc6b-4242-a720-43ba2b3c827e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:29.959115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:29.960953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:29.961587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:29.962004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:29.964255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:29.985439 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:29.989180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:30.018326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1222/2477] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:33.060521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2778-f69e-47df-ad2b-ed870ee8318d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:33.061010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2778-f69e-47df-ad2b-ed870ee8318d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:33.061103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2778-f69e-47df-ad2b-ed870ee8318d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:33.085013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-248f2778-f69e-47df-ad2b-ed870ee8318d None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:19:33.086332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-248f2778-f69e-47df-ad2b-ed870ee8318d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:33.142148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1256/2478] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:19:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:19:33.165791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:33.167255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:33.246613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385f58-5421-4604-9f77-d4d3e4ab7807 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:33.251703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a4e13-cbe5-486f-af05-85ce27fe08a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:33.252316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a4e13-cbe5-486f-af05-85ce27fe08a3 service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:33.252632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a4e13-cbe5-486f-af05-85ce27fe08a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:33.252904 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a4e13-cbe5-486f-af05-85ce27fe08a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:33.258723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:33.263007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a4e13-cbe5-486f-af05-85ce27fe08a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:33.265067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a4e13-cbe5-486f-af05-85ce27fe08a3 service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:33.274483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1223/2479] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:19:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:19:43.135849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=53 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:43.270176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:19:47.614260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f2778-f69e-47df-ad2b-ed870ee8318d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:47.629239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f2778-f69e-47df-ad2b-ed870ee8318d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:47.641333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:47.642402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:47.643228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:47.643322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:47.645218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:47.661097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:47.663573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:47.675540 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1257/2480] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:19:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:19:50.625821 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a6dfb-be9d-42c4-b8dc-cbfff1ffcc6b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:50.626314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a6dfb-be9d-42c4-b8dc-cbfff1ffcc6b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:50.626413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a6dfb-be9d-42c4-b8dc-cbfff1ffcc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:50.647517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-335a6dfb-be9d-42c4-b8dc-cbfff1ffcc6b None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:19:50.649448 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-335a6dfb-be9d-42c4-b8dc-cbfff1ffcc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:19:50.657785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1224/2481] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:19:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:19:50.669212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:19:50.683342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8861be-abbc-4e63-a105-536c947a5fce service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:19:50.690727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688d18f-c0ef-4e54-a094-ad32ad03b956 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:19:50.692256 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d18f-c0ef-4e54-a094-ad32ad03b956 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:19:50.692572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d18f-c0ef-4e54-a094-ad32ad03b956 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:19:50.692854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688d18f-c0ef-4e54-a094-ad32ad03b956 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:19:50.703734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688d18f-c0ef-4e54-a094-ad32ad03b956 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:19:50.705399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688d18f-c0ef-4e54-a094-ad32ad03b956 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:19:50.714465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1258/2482] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:19:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:20:00.641721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:00.710420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=12 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:05.478805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ba936-beaf-4122-b258-0e56b682e911 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:05.479475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ba936-beaf-4122-b258-0e56b682e911 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:05.479754 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ba936-beaf-4122-b258-0e56b682e911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:05.501632 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-381ba936-beaf-4122-b258-0e56b682e911 None None] MFA Rules not processed for user `f12cfb692b4b4947bc67a3199aa33e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:05.502935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-381ba936-beaf-4122-b258-0e56b682e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:05.560479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1225/2483] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:20:09.433492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2786-f994-4d15-8a10-771d6ef3244a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:09.433938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2786-f994-4d15-8a10-771d6ef3244a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:09.434036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2786-f994-4d15-8a10-771d6ef3244a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:09.454784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-aaad2786-f994-4d15-8a10-771d6ef3244a None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:09.455979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aaad2786-f994-4d15-8a10-771d6ef3244a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:09.463842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1259/2484] 10.222.0.70 () {58 vars in 967 bytes} [Tue Aug 9 19:20:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:20:09.476883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ba936-beaf-4122-b258-0e56b682e911 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:09.488466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ba936-beaf-4122-b258-0e56b682e911 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:09.493606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:09.494148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:09.494425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:09.494695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:09.502650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:09.504213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:09.512838 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1226/2485] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:20:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:20:19.450026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:19.509633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=31 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:23.461980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4334-077e-4ad8-9b7c-59b5930a1ab4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:23.462472 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4334-077e-4ad8-9b7c-59b5930a1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:23.462593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4334-077e-4ad8-9b7c-59b5930a1ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:23.481925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-daac4334-077e-4ad8-9b7c-59b5930a1ab4 None None] MFA Rules not processed for user `4f7899ab15b347b7be75e3433f0d5450`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:23.483078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-daac4334-077e-4ad8-9b7c-59b5930a1ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:23.490074 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1260/2486] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:20:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:20:33.094245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:33.095711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:33.152976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c42ce-2ba9-48db-96ec-c7ef03b2d4ef service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:33.156806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:33.157420 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:33.157698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:33.158001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:33.159713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:33.171497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:33.172961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:33.182949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1227/2487] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:20:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:20:33.476745 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:43.177895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=26 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:44.069817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49f71-941c-4c64-a1c8-44b16b88ea22 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:44.070296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49f71-941c-4c64-a1c8-44b16b88ea22 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:44.070407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49f71-941c-4c64-a1c8-44b16b88ea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:44.094343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-17b49f71-941c-4c64-a1c8-44b16b88ea22 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:44.095466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-17b49f71-941c-4c64-a1c8-44b16b88ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:44.103154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1261/2488] 10.222.0.70 () {58 vars in 967 bytes} [Tue Aug 9 19:20:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:20:44.127396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:44.142666 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924127f5-e0bd-43f5-9f83-8fa5e15d06af service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:44.148356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:44.149062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:44.149414 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:44.149912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:44.158539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:44.160199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:44.170487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1228/2489] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:20:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:20:49.203235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49f71-941c-4c64-a1c8-44b16b88ea22 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:49.205281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-17b49f71-941c-4c64-a1c8-44b16b88ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:49.227937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49f71-941c-4c64-a1c8-44b16b88ea22 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:49.233352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:49.233970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:49.234434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:49.234887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:49.236913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:49.249927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:49.251796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:49.264315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1262/2490] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:20:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:20:53.660022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:53.662101 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:53.681416 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf357a-1208-4af7-bf8c-9d6fa3ab5555 service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:53.686466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:53.687113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:53.687449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:53.687646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] PATH_INFO: `/v3/users/cbec728603924ccabfffc6cc8d18c5c9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:53.688801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbec728603924ccabfffc6cc8d18c5c9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:53.690764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:53.749971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbec728603924ccabfffc6cc8d18c5c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:53.760615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbec728603924ccabfffc6cc8d18c5c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:53.770205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbec728603924ccabfffc6cc8d18c5c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:53.770907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbec728603924ccabfffc6cc8d18c5c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:53.772187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbec728603924ccabfffc6cc8d18c5c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:53.793358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:53.797558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1229/2491] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:20:53 2022] DELETE /identity/v3/users/cbec728603924ccabfffc6cc8d18c5c9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:20:53.804388 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882a8ce-0374-4b84-aa63-f0289dad2f51 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:53.865738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:53.866356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:53.866638 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:53.866905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:53.868674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:53.945635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:53.947072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:53.958616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1263/2492] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:20:54.061643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:54.063204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:54.077463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689de5d1-717a-44a1-a893-427c4f022a82 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:54.083921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:54.094833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:54.095564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:54.096108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] PATH_INFO: `/v3/projects/63c5b147e9d0494381a36bd407f110db` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:54.097746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c5b147e9d0494381a36bd407f110db)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:54.100980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:54.177924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c5b147e9d0494381a36bd407f110db', 'request_id': 'req-030a8161-ee1b-437a-9801-a47e8174f7d6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.185067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c5b147e9d0494381a36bd407f110db', 'request_id': 'req-030a8161-ee1b-437a-9801-a47e8174f7d6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.199221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c5b147e9d0494381a36bd407f110db', 'request_id': 'req-030a8161-ee1b-437a-9801-a47e8174f7d6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.213067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] The token cache is being invalidate because project 63c5b147e9d0494381a36bd407f110db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:20:54.213650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.214476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.219306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1230/2493] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:20:54 2022] DELETE /identity/v3/projects/63c5b147e9d0494381a36bd407f110db => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:20:54.223240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:54.233970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a87ea5-bf45-43f8-b6fc-fb12942a0574 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:54.304311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:54.305295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:54.305759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:54.306350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] PATH_INFO: `/v3/users/2c0979f7d3c04ead9cd8b12a1b07cb33` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:54.308121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0979f7d3c04ead9cd8b12a1b07cb33)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:54.310831 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:54.352713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:54.369480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0979f7d3c04ead9cd8b12a1b07cb33'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.377260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0979f7d3c04ead9cd8b12a1b07cb33'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.394058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0979f7d3c04ead9cd8b12a1b07cb33'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.395125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0979f7d3c04ead9cd8b12a1b07cb33'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.395781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-04d97549-7951-4c52-8f25-3d00cbeb5339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0979f7d3c04ead9cd8b12a1b07cb33'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.419227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1264/2494] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:20:54 2022] DELETE /identity/v3/users/2c0979f7d3c04ead9cd8b12a1b07cb33 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:20:54.758648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:54.760154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:54.777716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a8161-ee1b-437a-9801-a47e8174f7d6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:54.840583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:54.869707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:54.870384 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:54.870665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:54.870950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] PATH_INFO: `/v3/projects/5e71118ace3e4342b88920494187374a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:54.872072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e71118ace3e4342b88920494187374a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:54.874092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:54.925833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e71118ace3e4342b88920494187374a', 'request_id': 'req-2714d092-96a8-4fe6-aaff-b4af007a2992'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.932721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e71118ace3e4342b88920494187374a', 'request_id': 'req-2714d092-96a8-4fe6-aaff-b4af007a2992'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.941882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e71118ace3e4342b88920494187374a', 'request_id': 'req-2714d092-96a8-4fe6-aaff-b4af007a2992'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.955563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] The token cache is being invalidate because project 5e71118ace3e4342b88920494187374a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:20:54.955699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.956364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2714d092-96a8-4fe6-aaff-b4af007a2992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:54.959758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1231/2495] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:20:54 2022] DELETE /identity/v3/projects/5e71118ace3e4342b88920494187374a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:20:54.965437 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569f0c-a162-4c31-a972-ea22fc7ead28 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:54.965897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569f0c-a162-4c31-a972-ea22fc7ead28 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:54.966053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569f0c-a162-4c31-a972-ea22fc7ead28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.006658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-07569f0c-a162-4c31-a972-ea22fc7ead28 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:55.009141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07569f0c-a162-4c31-a972-ea22fc7ead28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:55.098541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1265/2496] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:20:55.167205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2b11-eea0-4c97-8b3f-4e08cd8f04e3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.167572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2b11-eea0-4c97-8b3f-4e08cd8f04e3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.167768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2b11-eea0-4c97-8b3f-4e08cd8f04e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.190810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5a1f2b11-eea0-4c97-8b3f-4e08cd8f04e3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:55.193991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5a1f2b11-eea0-4c97-8b3f-4e08cd8f04e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:55.202390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1232/2497] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:20:55.206921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07569f0c-a162-4c31-a972-ea22fc7ead28 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:55.218325 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07569f0c-a162-4c31-a972-ea22fc7ead28 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:55.223343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:55.223933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.224230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.225001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.225220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:20:55.225516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:55.227103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:55.238989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1266/2498] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:20:55.243409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f2b11-eea0-4c97-8b3f-4e08cd8f04e3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:55.252519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f2b11-eea0-4c97-8b3f-4e08cd8f04e3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:55.257877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f693a-5410-4d4f-bad0-f14fb260cd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:55.258470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f693a-5410-4d4f-bad0-f14fb260cd8b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.258742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f693a-5410-4d4f-bad0-f14fb260cd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.259005 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f693a-5410-4d4f-bad0-f14fb260cd8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.259615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f693a-5410-4d4f-bad0-f14fb260cd8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:55.261557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f693a-5410-4d4f-bad0-f14fb260cd8b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:55.292801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1233/2499] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:20:55 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:20:55.304938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:55.315382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da01c-8318-40fa-a8de-a71acc544b99 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:55.375597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b304b54-9938-494b-8294-7c6ef7a76fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:55.376205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b304b54-9938-494b-8294-7c6ef7a76fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.376512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b304b54-9938-494b-8294-7c6ef7a76fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.376789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b304b54-9938-494b-8294-7c6ef7a76fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.377418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b304b54-9938-494b-8294-7c6ef7a76fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:55.379483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b304b54-9938-494b-8294-7c6ef7a76fc4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:55.406945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1267/2500] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:20:55 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:20:55.408157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f693a-5410-4d4f-bad0-f14fb260cd8b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:55.418455 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f693a-5410-4d4f-bad0-f14fb260cd8b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:55.423470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93943b34-d0be-40fd-80db-2085e4c645e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:55.424162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943b34-d0be-40fd-80db-2085e4c645e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.424542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943b34-d0be-40fd-80db-2085e4c645e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.424885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943b34-d0be-40fd-80db-2085e4c645e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.425603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93943b34-d0be-40fd-80db-2085e4c645e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:55.427818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93943b34-d0be-40fd-80db-2085e4c645e1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:55.438284 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1234/2501] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:20:55.442409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b304b54-9938-494b-8294-7c6ef7a76fc4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:55.453201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b304b54-9938-494b-8294-7c6ef7a76fc4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:55.457846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e1002-b1eb-4d69-91cc-205727ea1167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:55.459098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e1002-b1eb-4d69-91cc-205727ea1167 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.459456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e1002-b1eb-4d69-91cc-205727ea1167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.459793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e1002-b1eb-4d69-91cc-205727ea1167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.460533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e1002-b1eb-4d69-91cc-205727ea1167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:55.462162 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e1002-b1eb-4d69-91cc-205727ea1167 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:55.473154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1268/2502] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:20:55.478014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93943b34-d0be-40fd-80db-2085e4c645e1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:55.488037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93943b34-d0be-40fd-80db-2085e4c645e1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:55.493059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:55.493791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.494054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.494329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] PATH_INFO: `/v3/projects/57933942d654420985e7bc588dc98c1f/users/b66be721908a4eb1a136d416a4d8b2f9/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.510401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=57933942d654420985e7bc588dc98c1f, user_id=b66be721908a4eb1a136d416a4d8b2f9, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:55.513084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:55.563401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1235/2503] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:20:55 2022] PUT /identity/v3/projects/57933942d654420985e7bc588dc98c1f/users/b66be721908a4eb1a136d416a4d8b2f9/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:20:55.564847 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1b548-734b-4c5e-822d-49168ec26ddc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.565133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1b548-734b-4c5e-822d-49168ec26ddc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.565407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1b548-734b-4c5e-822d-49168ec26ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.587536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7cc1b548-734b-4c5e-822d-49168ec26ddc None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:55.588913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7cc1b548-734b-4c5e-822d-49168ec26ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:55.601106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:55.687931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1269/2504] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:20:55.694812 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:55.696503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:55.771928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc141-ed06-4bcf-bd9b-602a5332bdaf admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:55.776751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:55.777485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:55.777902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:55.778275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:55.790299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:55.791865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:55.804160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1236/2505] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:20:57.401585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1b548-734b-4c5e-822d-49168ec26ddc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:57.417504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1b548-734b-4c5e-822d-49168ec26ddc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:57.423089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:57.424276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:57.424666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:57.425005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:57.427239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:57.447734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:57.450224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:57.465293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1270/2506] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:20:58.151887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:58.154542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:58.180832 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b3e69-77ea-4e4d-b6ad-503dca35d7fa service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:58.204322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:58.205024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:58.205296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:58.205750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] PATH_INFO: `/v3/users/05147fb1762748ee9866ccc5bff4d514` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:58.207064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=05147fb1762748ee9866ccc5bff4d514)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:58.209598 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:58.222448 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:58.296786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05147fb1762748ee9866ccc5bff4d514'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:58.310006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05147fb1762748ee9866ccc5bff4d514'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:58.326090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05147fb1762748ee9866ccc5bff4d514'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:58.328810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05147fb1762748ee9866ccc5bff4d514'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:58.331318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1c0386de-28ce-45ed-ab78-643b2bfce6fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05147fb1762748ee9866ccc5bff4d514'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:58.357241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1237/2507] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:20:58 2022] DELETE /identity/v3/users/05147fb1762748ee9866ccc5bff4d514 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:20:58.980717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:58.991501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b653a47-ea30-497a-9543-bfb7e0eaefb9 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:59.127178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:59.127830 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:59.128108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:59.128376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] PATH_INFO: `/v3/projects/56cc04948356488f8722a735954fd650` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:59.129618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] RBAC: Authorizing `identity:delete_project(project_id=56cc04948356488f8722a735954fd650)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:59.132342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:59.234665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56cc04948356488f8722a735954fd650', 'request_id': 'req-578b66e2-9c02-4455-8996-461a4df0f01e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:59.247117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56cc04948356488f8722a735954fd650', 'request_id': 'req-578b66e2-9c02-4455-8996-461a4df0f01e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:59.270747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56cc04948356488f8722a735954fd650', 'request_id': 'req-578b66e2-9c02-4455-8996-461a4df0f01e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:59.295500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] The token cache is being invalidate because project 56cc04948356488f8722a735954fd650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:20:59.295849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:59.296495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-578b66e2-9c02-4455-8996-461a4df0f01e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:20:59.298627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1271/2508] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:20:58 2022] DELETE /identity/v3/projects/56cc04948356488f8722a735954fd650 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:20:59.304770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8339ed-8b17-422d-9d40-14e9fbd5bb6e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:59.305254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8339ed-8b17-422d-9d40-14e9fbd5bb6e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:59.305254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8339ed-8b17-422d-9d40-14e9fbd5bb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:59.349008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3b8339ed-8b17-422d-9d40-14e9fbd5bb6e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:59.350995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3b8339ed-8b17-422d-9d40-14e9fbd5bb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:59.494257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1238/2509] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:20:59.503392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b288c-374d-4ed3-b4ad-91092810b978 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:59.503970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b288c-374d-4ed3-b4ad-91092810b978 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:59.504482 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b288c-374d-4ed3-b4ad-91092810b978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:59.551584 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-734b288c-374d-4ed3-b4ad-91092810b978 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:20:59.553578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-734b288c-374d-4ed3-b4ad-91092810b978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:20:59.577395 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1272/2510] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:20:59.581437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8339ed-8b17-422d-9d40-14e9fbd5bb6e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:59.601687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8339ed-8b17-422d-9d40-14e9fbd5bb6e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:59.617637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:59.618615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:59.619125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:59.619597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:59.620483 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:20:59.621222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:59.623257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:59.651865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1239/2511] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:20:59.656263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b288c-374d-4ed3-b4ad-91092810b978 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:59.674240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b288c-374d-4ed3-b4ad-91092810b978 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:59.681081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2f7b-8694-4359-9a24-04705967d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:59.682062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2f7b-8694-4359-9a24-04705967d931 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:59.682536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2f7b-8694-4359-9a24-04705967d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:59.683024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2f7b-8694-4359-9a24-04705967d931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:59.684123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2f7b-8694-4359-9a24-04705967d931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:59.686104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2f7b-8694-4359-9a24-04705967d931 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:59.765212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1273/2512] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:20:59 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:20:59.769201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:59.788176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73846e7-e8cc-426e-aede-73007b946315 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:20:59.918222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c541b-f778-4fc0-bda3-b2185fee272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:20:59.919117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c541b-f778-4fc0-bda3-b2185fee272b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:20:59.919401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c541b-f778-4fc0-bda3-b2185fee272b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:20:59.919672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c541b-f778-4fc0-bda3-b2185fee272b admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:20:59.920761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c541b-f778-4fc0-bda3-b2185fee272b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:20:59.923086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c541b-f778-4fc0-bda3-b2185fee272b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:20:59.964478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:59.975374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1240/2513] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:20:59 2022] POST /identity/v3/users => generated 279 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:20:59.976244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2f7b-8694-4359-9a24-04705967d931 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:20:59.985267 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:20:59.991339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2f7b-8694-4359-9a24-04705967d931 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:00.019590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e18050-244f-4306-b9c8-0d0e953a75ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:00.020373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e18050-244f-4306-b9c8-0d0e953a75ec admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:00.020809 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e18050-244f-4306-b9c8-0d0e953a75ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:00.021345 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e18050-244f-4306-b9c8-0d0e953a75ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:00.022448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e18050-244f-4306-b9c8-0d0e953a75ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:00.024346 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e18050-244f-4306-b9c8-0d0e953a75ec admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:00.067309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1274/2514] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:00.072899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c541b-f778-4fc0-bda3-b2185fee272b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:00.093884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c541b-f778-4fc0-bda3-b2185fee272b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:00.103365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7d019-992f-46b3-82f8-7dc6a4e69979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:00.104120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7d019-992f-46b3-82f8-7dc6a4e69979 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:00.104400 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7d019-992f-46b3-82f8-7dc6a4e69979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:00.105107 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7d019-992f-46b3-82f8-7dc6a4e69979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:00.105533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7d019-992f-46b3-82f8-7dc6a4e69979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:00.107157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7d019-992f-46b3-82f8-7dc6a4e69979 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:00.123423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1241/2515] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:00.129058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e18050-244f-4306-b9c8-0d0e953a75ec admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:00.147970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e18050-244f-4306-b9c8-0d0e953a75ec admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:00.160431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f475f7-b189-4957-ae7f-3e82ab06b216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:00.161294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f475f7-b189-4957-ae7f-3e82ab06b216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:00.161807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f475f7-b189-4957-ae7f-3e82ab06b216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:00.162353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f475f7-b189-4957-ae7f-3e82ab06b216 admin admin] PATH_INFO: `/v3/projects/b5cf841803c3483ba23a28309b25a8d5/users/5f58617086df4c62a4593ee61ae26409/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:00.194874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f475f7-b189-4957-ae7f-3e82ab06b216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5cf841803c3483ba23a28309b25a8d5, user_id=5f58617086df4c62a4593ee61ae26409, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:00.201019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f475f7-b189-4957-ae7f-3e82ab06b216 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:00.225397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1275/2516] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:00 2022] PUT /identity/v3/projects/b5cf841803c3483ba23a28309b25a8d5/users/5f58617086df4c62a4593ee61ae26409/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:00.235998 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c3ea9-4aa1-4eeb-aad6-877f8a27ae8e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:00.236293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c3ea9-4aa1-4eeb-aad6-877f8a27ae8e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:00.236579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c3ea9-4aa1-4eeb-aad6-877f8a27ae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:00.276349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-314c3ea9-4aa1-4eeb-aad6-877f8a27ae8e None None] MFA Rules not processed for user `5f58617086df4c62a4593ee61ae26409`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:00.278095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-314c3ea9-4aa1-4eeb-aad6-877f8a27ae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:00.369843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f475f7-b189-4957-ae7f-3e82ab06b216 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:00.371292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1242/2517] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:00.493812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f475f7-b189-4957-ae7f-3e82ab06b216 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:00.499978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c93d4-ab54-4222-8bbc-1a637ad37ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:00.500598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c93d4-ab54-4222-8bbc-1a637ad37ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:00.502228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c93d4-ab54-4222-8bbc-1a637ad37ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:00.502763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c93d4-ab54-4222-8bbc-1a637ad37ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:00.503814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c93d4-ab54-4222-8bbc-1a637ad37ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:00.506851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c93d4-ab54-4222-8bbc-1a637ad37ce5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:00.559734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1276/2518] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:00 2022] POST /identity/v3/projects => generated 355 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:00.564165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c3ea9-4aa1-4eeb-aad6-877f8a27ae8e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:00.579373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c3ea9-4aa1-4eeb-aad6-877f8a27ae8e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:00.731306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b13ce0-85a1-4f51-aa59-2930bffc2112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:00.732285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ce0-85a1-4f51-aa59-2930bffc2112 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:00.732760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ce0-85a1-4f51-aa59-2930bffc2112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:00.733208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ce0-85a1-4f51-aa59-2930bffc2112 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:00.734480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b13ce0-85a1-4f51-aa59-2930bffc2112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:00.736638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b13ce0-85a1-4f51-aa59-2930bffc2112 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:00.778459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1243/2519] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:00 2022] POST /identity/v3/users => generated 284 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:00.782121 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c93d4-ab54-4222-8bbc-1a637ad37ce5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:00.797192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c93d4-ab54-4222-8bbc-1a637ad37ce5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:00.802953 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93feaf-01b8-4ec8-a97f-fda9b492169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:00.803571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93feaf-01b8-4ec8-a97f-fda9b492169a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:00.803908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93feaf-01b8-4ec8-a97f-fda9b492169a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:00.804312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93feaf-01b8-4ec8-a97f-fda9b492169a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:00.805261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93feaf-01b8-4ec8-a97f-fda9b492169a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:00.807665 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93feaf-01b8-4ec8-a97f-fda9b492169a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:00.829194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1277/2520] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:00.829419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b13ce0-85a1-4f51-aa59-2930bffc2112 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:00.845643 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b13ce0-85a1-4f51-aa59-2930bffc2112 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:00.853227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5018f22-0cf5-46e3-b1c8-e7b67225fd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:00.854125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018f22-0cf5-46e3-b1c8-e7b67225fd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:00.854592 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018f22-0cf5-46e3-b1c8-e7b67225fd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:00.855114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018f22-0cf5-46e3-b1c8-e7b67225fd41 admin admin] PATH_INFO: `/v3/projects/e8b03732e5ad479d859a7de3fd59aee5/users/6fabe84e22354cd7848d54046d862cb1/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:00.882797 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5018f22-0cf5-46e3-b1c8-e7b67225fd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8b03732e5ad479d859a7de3fd59aee5, user_id=6fabe84e22354cd7848d54046d862cb1, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:00.885809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5018f22-0cf5-46e3-b1c8-e7b67225fd41 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:00.917246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1244/2521] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:00 2022] PUT /identity/v3/projects/e8b03732e5ad479d859a7de3fd59aee5/users/6fabe84e22354cd7848d54046d862cb1/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:00.922296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93feaf-01b8-4ec8-a97f-fda9b492169a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:00.952656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93feaf-01b8-4ec8-a97f-fda9b492169a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:01.107340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90f1e4-af9d-4961-9674-2c9821d2d1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:01.108013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90f1e4-af9d-4961-9674-2c9821d2d1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:01.108251 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90f1e4-af9d-4961-9674-2c9821d2d1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:01.108552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90f1e4-af9d-4961-9674-2c9821d2d1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:01.109205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90f1e4-af9d-4961-9674-2c9821d2d1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:01.110827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90f1e4-af9d-4961-9674-2c9821d2d1b5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:01.141684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5018f22-0cf5-46e3-b1c8-e7b67225fd41 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:01.142300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1278/2522] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:01.162606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5018f22-0cf5-46e3-b1c8-e7b67225fd41 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:01.181505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:01.182361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:01.182490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:01.182706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] PATH_INFO: `/v3/projects/e8b03732e5ad479d859a7de3fd59aee5/users/6fabe84e22354cd7848d54046d862cb1/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:01.188096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8b03732e5ad479d859a7de3fd59aee5, user_id=6fabe84e22354cd7848d54046d862cb1, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:01.198324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:01.232710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1245/2523] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:01 2022] PUT /identity/v3/projects/e8b03732e5ad479d859a7de3fd59aee5/users/6fabe84e22354cd7848d54046d862cb1/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:01.241690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cfc3d-ddd6-487a-b827-0638f77ef156 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:01.242291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cfc3d-ddd6-487a-b827-0638f77ef156 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:01.242771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cfc3d-ddd6-487a-b827-0638f77ef156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:01.278605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8c3cfc3d-ddd6-487a-b827-0638f77ef156 None None] MFA Rules not processed for user `6fabe84e22354cd7848d54046d862cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:01.280272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8c3cfc3d-ddd6-487a-b827-0638f77ef156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:01.327386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:01.414592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1279/2524] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:01.429009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:01.431432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:01.527836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:01.604091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425c3fc-5447-4187-a34b-5d6a9b0a7679 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:01.612098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f057c-18f0-4943-95ad-62c0d16e8e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:01.612769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f057c-18f0-4943-95ad-62c0d16e8e98 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:01.613048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f057c-18f0-4943-95ad-62c0d16e8e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:01.613322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f057c-18f0-4943-95ad-62c0d16e8e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:01.642251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f057c-18f0-4943-95ad-62c0d16e8e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:01.643745 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f057c-18f0-4943-95ad-62c0d16e8e98 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:01.655215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1246/2525] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:01.789447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cfc3d-ddd6-487a-b827-0638f77ef156 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:01.812273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cfc3d-ddd6-487a-b827-0638f77ef156 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:01.814230 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eecee7-9c26-415f-8972-4525dd0a21da None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:01.814731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eecee7-9c26-415f-8972-4525dd0a21da None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:01.815164 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eecee7-9c26-415f-8972-4525dd0a21da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:01.816017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:01.816657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:01.816936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:01.817199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:01.876416 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e2eecee7-9c26-415f-8972-4525dd0a21da None None] MFA Rules not processed for user `b66be721908a4eb1a136d416a4d8b2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:01.879173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e2eecee7-9c26-415f-8972-4525dd0a21da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:01.932611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:01.934135 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:01.946378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1280/2526] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:02.017422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1247/2527] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:02.018147 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:02.019597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:02.139843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67163e3-9ea8-482e-8a9f-a658a636715c service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:02.145567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d725a3-5557-42a6-a582-177924a8cf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:02.146607 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d725a3-5557-42a6-a582-177924a8cf28 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:02.147040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d725a3-5557-42a6-a582-177924a8cf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:02.147465 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d725a3-5557-42a6-a582-177924a8cf28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:02.148409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d725a3-5557-42a6-a582-177924a8cf28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:02.150649 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d725a3-5557-42a6-a582-177924a8cf28 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:02.190378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1281/2528] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:02 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:02.200302 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eecee7-9c26-415f-8972-4525dd0a21da None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:02.215124 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eecee7-9c26-415f-8972-4525dd0a21da None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:02.286868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9251ac-263b-4698-af66-8c825dbbbb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:02.287555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9251ac-263b-4698-af66-8c825dbbbb91 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:02.287925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9251ac-263b-4698-af66-8c825dbbbb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:02.288105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9251ac-263b-4698-af66-8c825dbbbb91 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:02.288750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9251ac-263b-4698-af66-8c825dbbbb91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:02.290809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9251ac-263b-4698-af66-8c825dbbbb91 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:02.320815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1248/2529] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:02 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:02.322419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d725a3-5557-42a6-a582-177924a8cf28 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:02.335798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d725a3-5557-42a6-a582-177924a8cf28 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:02.342603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8b0d4-c98b-4c63-9c63-a3a9ad828903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:02.343454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8b0d4-c98b-4c63-9c63-a3a9ad828903 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:02.343924 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8b0d4-c98b-4c63-9c63-a3a9ad828903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:02.344975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8b0d4-c98b-4c63-9c63-a3a9ad828903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:02.345704 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8b0d4-c98b-4c63-9c63-a3a9ad828903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:02.347419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8b0d4-c98b-4c63-9c63-a3a9ad828903 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:02.374047 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9251ac-263b-4698-af66-8c825dbbbb91 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:02.374276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1282/2530] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:02.386318 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9251ac-263b-4698-af66-8c825dbbbb91 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:02.392239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4168d-c251-473d-b615-a8104a72f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:02.394248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4168d-c251-473d-b615-a8104a72f53e admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:02.395428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4168d-c251-473d-b615-a8104a72f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:02.395974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4168d-c251-473d-b615-a8104a72f53e admin admin] PATH_INFO: `/v3/projects/f90a8128151c4216bf2fc878233ce45a/users/40e5ac7c8b334a46a2bbe2121c8f5566/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:02.416191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4168d-c251-473d-b615-a8104a72f53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90a8128151c4216bf2fc878233ce45a, user_id=40e5ac7c8b334a46a2bbe2121c8f5566, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:02.419536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4168d-c251-473d-b615-a8104a72f53e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:02.437610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1249/2531] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:02 2022] PUT /identity/v3/projects/f90a8128151c4216bf2fc878233ce45a/users/40e5ac7c8b334a46a2bbe2121c8f5566/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:02.441827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8b0d4-c98b-4c63-9c63-a3a9ad828903 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:02.456900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8b0d4-c98b-4c63-9c63-a3a9ad828903 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:02.530680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:02.566786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:02.567547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:02.568494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:02.570492 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:02.571372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:02.574367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:02.595148 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4168d-c251-473d-b615-a8104a72f53e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:02.596933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1283/2532] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:02.618702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4168d-c251-473d-b615-a8104a72f53e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:02.623703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:02.624371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:02.624685 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:02.624956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] PATH_INFO: `/v3/projects/f90a8128151c4216bf2fc878233ce45a/users/40e5ac7c8b334a46a2bbe2121c8f5566/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:02.626928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90a8128151c4216bf2fc878233ce45a, user_id=40e5ac7c8b334a46a2bbe2121c8f5566, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:02.630006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:02.644563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1250/2533] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:02 2022] PUT /identity/v3/projects/f90a8128151c4216bf2fc878233ce45a/users/40e5ac7c8b334a46a2bbe2121c8f5566/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:03.198235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:03.201242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:03.218500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb1f0c-d285-4187-a3ed-79c38e6f6025 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:03.317146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:03.320035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:03.320432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:03.320962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] PATH_INFO: `/v3/users/e9ad80a549b14b25ac71cd96832a74e7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:03.322575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ad80a549b14b25ac71cd96832a74e7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:03.326566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:03.438994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ad80a549b14b25ac71cd96832a74e7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:03.452587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ad80a549b14b25ac71cd96832a74e7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:03.484763 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ad80a549b14b25ac71cd96832a74e7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:03.486730 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ad80a549b14b25ac71cd96832a74e7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:03.488298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-83ee4ea2-96b1-4075-942f-f3a632f12d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ad80a549b14b25ac71cd96832a74e7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:03.551702 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1284/2534] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:03 2022] DELETE /identity/v3/users/e9ad80a549b14b25ac71cd96832a74e7 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:03.799571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:03.802669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:03.827125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731758a-ccb5-49f2-b1fb-fae259f40f84 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:03.844710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:03.911636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:03.912716 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:03.913170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:03.913446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] PATH_INFO: `/v3/projects/8b32d5c45c614fd08a57af6be1143590` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:03.914689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b32d5c45c614fd08a57af6be1143590)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:03.916696 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:03.986511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b32d5c45c614fd08a57af6be1143590', 'request_id': 'req-fb56be86-f3e7-470f-b309-47a86e90eec8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:03.997200 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b32d5c45c614fd08a57af6be1143590', 'request_id': 'req-fb56be86-f3e7-470f-b309-47a86e90eec8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:04.008560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b32d5c45c614fd08a57af6be1143590', 'request_id': 'req-fb56be86-f3e7-470f-b309-47a86e90eec8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:04.023526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] The token cache is being invalidate because project 8b32d5c45c614fd08a57af6be1143590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:04.024430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:04.025249 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fb56be86-f3e7-470f-b309-47a86e90eec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:04.027155 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1251/2535] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:03 2022] DELETE /identity/v3/projects/8b32d5c45c614fd08a57af6be1143590 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:04.032826 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5cfb5-f481-4ea7-8f20-72f7e2d7084b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.033261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5cfb5-f481-4ea7-8f20-72f7e2d7084b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.033367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5cfb5-f481-4ea7-8f20-72f7e2d7084b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.061939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-56d5cfb5-f481-4ea7-8f20-72f7e2d7084b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:04.063913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-56d5cfb5-f481-4ea7-8f20-72f7e2d7084b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:04.124973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:04.146413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1285/2536] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:04.155270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a212-26a4-4e8a-9dcd-8fad162d0d62 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.155564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a212-26a4-4e8a-9dcd-8fad162d0d62 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.155819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a212-26a4-4e8a-9dcd-8fad162d0d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.179532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-10a8a212-26a4-4e8a-9dcd-8fad162d0d62 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:04.180690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-10a8a212-26a4-4e8a-9dcd-8fad162d0d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:04.190073 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1252/2537] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:04.195387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5cfb5-f481-4ea7-8f20-72f7e2d7084b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:04.210237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5cfb5-f481-4ea7-8f20-72f7e2d7084b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:04.216905 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:04.217172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.217440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.217712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.218469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:04.219189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:04.220424 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:04.234441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1286/2538] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:21:04.238712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8a212-26a4-4e8a-9dcd-8fad162d0d62 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:04.252501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8a212-26a4-4e8a-9dcd-8fad162d0d62 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:04.259079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b4db4-2de3-446f-8f9f-77b1378621bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:04.259705 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b4db4-2de3-446f-8f9f-77b1378621bd admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.259976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b4db4-2de3-446f-8f9f-77b1378621bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.260245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b4db4-2de3-446f-8f9f-77b1378621bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.260983 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b4db4-2de3-446f-8f9f-77b1378621bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:04.262961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b4db4-2de3-446f-8f9f-77b1378621bd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:04.298985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1253/2539] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:04 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:04.303597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:04.314231 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21368f-2e4d-4b85-a71a-61237b50a8ff admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:04.385221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d9a95-9b72-4f10-99e5-74f334b55a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:04.386272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d9a95-9b72-4f10-99e5-74f334b55a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.387036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d9a95-9b72-4f10-99e5-74f334b55a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.388778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d9a95-9b72-4f10-99e5-74f334b55a24 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.389557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d9a95-9b72-4f10-99e5-74f334b55a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:04.391404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d9a95-9b72-4f10-99e5-74f334b55a24 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:04.421282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1287/2540] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:04 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:04.425606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b4db4-2de3-446f-8f9f-77b1378621bd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:04.436712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b4db4-2de3-446f-8f9f-77b1378621bd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:04.442263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fadff-fd43-473d-b973-f973f43a8684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:04.442967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fadff-fd43-473d-b973-f973f43a8684 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.443245 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fadff-fd43-473d-b973-f973f43a8684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.443508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fadff-fd43-473d-b973-f973f43a8684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.444157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fadff-fd43-473d-b973-f973f43a8684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:04.445853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fadff-fd43-473d-b973-f973f43a8684 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:04.457260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1254/2541] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:21:04.462068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d9a95-9b72-4f10-99e5-74f334b55a24 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:04.476947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d9a95-9b72-4f10-99e5-74f334b55a24 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:04.484908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452073a4-efdc-4c7e-b918-6b100f0b8143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:04.485846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452073a4-efdc-4c7e-b918-6b100f0b8143 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.486328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452073a4-efdc-4c7e-b918-6b100f0b8143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.486658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452073a4-efdc-4c7e-b918-6b100f0b8143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.487401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452073a4-efdc-4c7e-b918-6b100f0b8143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:04.489197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452073a4-efdc-4c7e-b918-6b100f0b8143 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:04.502617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1288/2542] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:21:04.507743 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fadff-fd43-473d-b973-f973f43a8684 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:04.519255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fadff-fd43-473d-b973-f973f43a8684 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:04.525630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:04.526588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.526916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.527544 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] PATH_INFO: `/v3/projects/bda36c8a4ba04263a908f7d328e81334/users/e16f8c4c7d9a4b34bdc4cd75a4b8ea22/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.555312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda36c8a4ba04263a908f7d328e81334, user_id=e16f8c4c7d9a4b34bdc4cd75a4b8ea22, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:04.559622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:04.592966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1255/2543] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:04 2022] PUT /identity/v3/projects/bda36c8a4ba04263a908f7d328e81334/users/e16f8c4c7d9a4b34bdc4cd75a4b8ea22/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:04.602152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08079e-0159-4d75-81d4-9c4dd02b9748 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:04.602440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08079e-0159-4d75-81d4-9c4dd02b9748 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:04.602710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08079e-0159-4d75-81d4-9c4dd02b9748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:04.642617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-da08079e-0159-4d75-81d4-9c4dd02b9748 None None] MFA Rules not processed for user `e16f8c4c7d9a4b34bdc4cd75a4b8ea22`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:04.643303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-da08079e-0159-4d75-81d4-9c4dd02b9748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:04.777621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1289/2544] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:04.790427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:04.797218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:04.909321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:05.048262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d79c53-5045-427a-a0e9-9a98a62c4e59 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:05.064153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b79a4d-5ad8-422b-a812-a08e31b75c35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:05.065035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79a4d-5ad8-422b-a812-a08e31b75c35 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:05.065476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79a4d-5ad8-422b-a812-a08e31b75c35 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:05.066049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b79a4d-5ad8-422b-a812-a08e31b75c35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:05.076995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b79a4d-5ad8-422b-a812-a08e31b75c35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:05.079388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b79a4d-5ad8-422b-a812-a08e31b75c35 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:05.109488 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1256/2545] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:08.322893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b307e3-cf16-41cd-a81f-ef6d3f9104cc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:08.322893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b307e3-cf16-41cd-a81f-ef6d3f9104cc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:08.322893 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b307e3-cf16-41cd-a81f-ef6d3f9104cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:08.323553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b79a4d-5ad8-422b-a812-a08e31b75c35 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:08.397317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-97b307e3-cf16-41cd-a81f-ef6d3f9104cc None None] MFA Rules not processed for user `40e5ac7c8b334a46a2bbe2121c8f5566`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:08.399141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-97b307e3-cf16-41cd-a81f-ef6d3f9104cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:08.413749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b79a4d-5ad8-422b-a812-a08e31b75c35 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:08.418789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:08.419501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:08.419691 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:08.420227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] PATH_INFO: `/v3/users/e16f8c4c7d9a4b34bdc4cd75a4b8ea22` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:08.422834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e16f8c4c7d9a4b34bdc4cd75a4b8ea22)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:08.425452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:08.463104 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:08.506228 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e16f8c4c7d9a4b34bdc4cd75a4b8ea22'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:08.510530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1290/2546] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:08.515785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e16f8c4c7d9a4b34bdc4cd75a4b8ea22'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:08.526123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e16f8c4c7d9a4b34bdc4cd75a4b8ea22'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:08.527183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e16f8c4c7d9a4b34bdc4cd75a4b8ea22'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:08.528152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c16a5427-78f0-4325-866f-a493acafd129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e16f8c4c7d9a4b34bdc4cd75a4b8ea22'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:08.530303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b307e3-cf16-41cd-a81f-ef6d3f9104cc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:08.532011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-97b307e3-cf16-41cd-a81f-ef6d3f9104cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:08.557022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b307e3-cf16-41cd-a81f-ef6d3f9104cc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:08.557558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1257/2547] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:08 2022] DELETE /identity/v3/users/e16f8c4c7d9a4b34bdc4cd75a4b8ea22 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:08.561903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4fb2-5547-4db9-8eb2-3728baab9a42 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:08.562828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4fb2-5547-4db9-8eb2-3728baab9a42 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:08.562828 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4fb2-5547-4db9-8eb2-3728baab9a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:08.589513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-72ff4fb2-5547-4db9-8eb2-3728baab9a42 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:08.590733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-72ff4fb2-5547-4db9-8eb2-3728baab9a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:08.635932 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:08.636589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:08.636870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:08.637276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:08.637398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:08.638968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:08.658533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1258/2548] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:08.679095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff4fb2-5547-4db9-8eb2-3728baab9a42 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:08.681088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-72ff4fb2-5547-4db9-8eb2-3728baab9a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:08.736516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:08.738685 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:08.748944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1291/2549] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:08.759300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff4fb2-5547-4db9-8eb2-3728baab9a42 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:08.763160 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523128-5216-4cd6-8c6c-a0e7671b6949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:08.763776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523128-5216-4cd6-8c6c-a0e7671b6949 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:08.764055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523128-5216-4cd6-8c6c-a0e7671b6949 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:08.764322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523128-5216-4cd6-8c6c-a0e7671b6949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:08.775537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523128-5216-4cd6-8c6c-a0e7671b6949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:08.777035 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523128-5216-4cd6-8c6c-a0e7671b6949 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:08.788212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1259/2550] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:09.154114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:09.156393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:09.183308 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239c7c-78c1-4b3f-ac0b-a19a47e31759 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:09.191189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:09.192317 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.193580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.194032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] PATH_INFO: `/v3/projects/bda36c8a4ba04263a908f7d328e81334` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.195696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda36c8a4ba04263a908f7d328e81334)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:09.198219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:09.266525 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda36c8a4ba04263a908f7d328e81334', 'request_id': 'req-620626c7-b47e-43cd-a26c-512b769b3b45'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:09.275179 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda36c8a4ba04263a908f7d328e81334', 'request_id': 'req-620626c7-b47e-43cd-a26c-512b769b3b45'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:09.308629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda36c8a4ba04263a908f7d328e81334', 'request_id': 'req-620626c7-b47e-43cd-a26c-512b769b3b45'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:09.309797 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:09.330816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] The token cache is being invalidate because project bda36c8a4ba04263a908f7d328e81334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:09.331474 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:09.331963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-620626c7-b47e-43cd-a26c-512b769b3b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:09.337193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1292/2551] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:09 2022] DELETE /identity/v3/projects/bda36c8a4ba04263a908f7d328e81334 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:09.341040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98837c7-27dd-481c-bf42-bf249def6b9d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.341470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98837c7-27dd-481c-bf42-bf249def6b9d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.341651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98837c7-27dd-481c-bf42-bf249def6b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.372726 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c98837c7-27dd-481c-bf42-bf249def6b9d None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:09.374137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c98837c7-27dd-481c-bf42-bf249def6b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:09.453723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1260/2552] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:09.462178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e32f6d-3684-44f6-85b9-5b61ea45ad17 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.462461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e32f6d-3684-44f6-85b9-5b61ea45ad17 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.462736 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e32f6d-3684-44f6-85b9-5b61ea45ad17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.485405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-09e32f6d-3684-44f6-85b9-5b61ea45ad17 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:09.486874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-09e32f6d-3684-44f6-85b9-5b61ea45ad17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:09.501264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1293/2553] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:09.506020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98837c7-27dd-481c-bf42-bf249def6b9d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:09.520201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98837c7-27dd-481c-bf42-bf249def6b9d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:09.529780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:09.530590 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.530969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.531372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.532105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:09.532515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:09.534387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:09.555849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1261/2554] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:21:09.560480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e32f6d-3684-44f6-85b9-5b61ea45ad17 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:09.570818 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e32f6d-3684-44f6-85b9-5b61ea45ad17 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:09.578320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17644c3d-2153-48ae-ad13-b92ef249015c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:09.578986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644c3d-2153-48ae-ad13-b92ef249015c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.579337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644c3d-2153-48ae-ad13-b92ef249015c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.579776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644c3d-2153-48ae-ad13-b92ef249015c admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.580453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17644c3d-2153-48ae-ad13-b92ef249015c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:09.582446 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17644c3d-2153-48ae-ad13-b92ef249015c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:09.624338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1294/2555] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:09 2022] POST /identity/v3/projects => generated 405 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:09.624966 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:09.636158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cde2b1-dde9-4b14-a221-116767495af3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:09.720340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e365b-f473-4955-9699-f1838510eeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:09.721261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e365b-f473-4955-9699-f1838510eeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.721780 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e365b-f473-4955-9699-f1838510eeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.722298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e365b-f473-4955-9699-f1838510eeb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.723519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e365b-f473-4955-9699-f1838510eeb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:09.726339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e365b-f473-4955-9699-f1838510eeb2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:09.768486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1262/2556] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:09 2022] POST /identity/v3/users => generated 303 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:09.771371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17644c3d-2153-48ae-ad13-b92ef249015c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:09.783022 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17644c3d-2153-48ae-ad13-b92ef249015c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:09.789793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099a1dd-dffa-4779-8e6f-ee894c74fb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:09.791105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099a1dd-dffa-4779-8e6f-ee894c74fb1e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.791393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099a1dd-dffa-4779-8e6f-ee894c74fb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.791836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099a1dd-dffa-4779-8e6f-ee894c74fb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.792756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099a1dd-dffa-4779-8e6f-ee894c74fb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:09.794613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099a1dd-dffa-4779-8e6f-ee894c74fb1e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:09.806841 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1295/2557] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:09.811129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e365b-f473-4955-9699-f1838510eeb2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:09.822255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e365b-f473-4955-9699-f1838510eeb2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:09.829327 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1191b9-d3a0-4dc4-a44c-0d3c15d44df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:09.831816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1191b9-d3a0-4dc4-a44c-0d3c15d44df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.832259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1191b9-d3a0-4dc4-a44c-0d3c15d44df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.833763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1191b9-d3a0-4dc4-a44c-0d3c15d44df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.834695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1191b9-d3a0-4dc4-a44c-0d3c15d44df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:09.836876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1191b9-d3a0-4dc4-a44c-0d3c15d44df0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:09.840601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:09.870527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1263/2558] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:09.875451 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099a1dd-dffa-4779-8e6f-ee894c74fb1e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:09.888567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099a1dd-dffa-4779-8e6f-ee894c74fb1e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:09.896186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b6414-507a-47bc-a6e8-deb90638f7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:09.897371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6414-507a-47bc-a6e8-deb90638f7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.897864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6414-507a-47bc-a6e8-deb90638f7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.898341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6414-507a-47bc-a6e8-deb90638f7ec admin admin] PATH_INFO: `/v3/projects/129dde02e7f44596882782379d377a50/users/901c78db6c0343cba1205de57c25b6d5/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:09.933832 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b6414-507a-47bc-a6e8-deb90638f7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=129dde02e7f44596882782379d377a50, user_id=901c78db6c0343cba1205de57c25b6d5, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:09.937497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b6414-507a-47bc-a6e8-deb90638f7ec admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:09.977691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1296/2559] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:09 2022] PUT /identity/v3/projects/129dde02e7f44596882782379d377a50/users/901c78db6c0343cba1205de57c25b6d5/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:09.981209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbcb82-7ff2-4364-b14c-8d447455d876 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:09.981769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbcb82-7ff2-4364-b14c-8d447455d876 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:09.982750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbcb82-7ff2-4364-b14c-8d447455d876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:10.010368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4afbcb82-7ff2-4364-b14c-8d447455d876 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:10.013646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4afbcb82-7ff2-4364-b14c-8d447455d876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:10.118248 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f8893-f308-4a27-bfe6-2fcbc1ce9b52 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:10.118615 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f8893-f308-4a27-bfe6-2fcbc1ce9b52 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:10.119202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f8893-f308-4a27-bfe6-2fcbc1ce9b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:10.146994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1264/2560] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:10.157044 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b68f8893-f308-4a27-bfe6-2fcbc1ce9b52 None None] MFA Rules not processed for user `b66be721908a4eb1a136d416a4d8b2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:10.158595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbcb82-7ff2-4364-b14c-8d447455d876 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:10.160854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b68f8893-f308-4a27-bfe6-2fcbc1ce9b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:10.163210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4afbcb82-7ff2-4364-b14c-8d447455d876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:10.234065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbcb82-7ff2-4364-b14c-8d447455d876 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:10.237833 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c87cb9-4d12-4abe-82c1-df99af33704a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:10.238447 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c87cb9-4d12-4abe-82c1-df99af33704a service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:10.238725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c87cb9-4d12-4abe-82c1-df99af33704a service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:10.238993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c87cb9-4d12-4abe-82c1-df99af33704a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:10.240547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1297/2561] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:10.249508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c87cb9-4d12-4abe-82c1-df99af33704a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:10.250929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c87cb9-4d12-4abe-82c1-df99af33704a service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:10.262741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1265/2562] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:11.029850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3c6a-70d8-4b44-af21-ef0f2cf0c134 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:11.031130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3c6a-70d8-4b44-af21-ef0f2cf0c134 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:11.031660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3c6a-70d8-4b44-af21-ef0f2cf0c134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:11.069558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b1db3c6a-70d8-4b44-af21-ef0f2cf0c134 None None] MFA Rules not processed for user `901c78db6c0343cba1205de57c25b6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:11.071670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b1db3c6a-70d8-4b44-af21-ef0f2cf0c134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:11.198038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c87cb9-4d12-4abe-82c1-df99af33704a service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:11.204096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1298/2563] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:11.211484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c87cb9-4d12-4abe-82c1-df99af33704a service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:11.217563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0e8d3-09c2-47ed-9708-82cc1d9fbac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:11.218581 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0e8d3-09c2-47ed-9708-82cc1d9fbac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:11.218967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0e8d3-09c2-47ed-9708-82cc1d9fbac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:11.219334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0e8d3-09c2-47ed-9708-82cc1d9fbac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:11.220916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0e8d3-09c2-47ed-9708-82cc1d9fbac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:11.222956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0e8d3-09c2-47ed-9708-82cc1d9fbac0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:11.269746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1266/2564] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:11 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:11.271829 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db3c6a-70d8-4b44-af21-ef0f2cf0c134 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:11.287002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db3c6a-70d8-4b44-af21-ef0f2cf0c134 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:11.321216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:11.401622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91496bd4-682d-4426-96f7-4986abc4261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:11.402252 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496bd4-682d-4426-96f7-4986abc4261d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:11.402519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496bd4-682d-4426-96f7-4986abc4261d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:11.402789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496bd4-682d-4426-96f7-4986abc4261d admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:11.403425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91496bd4-682d-4426-96f7-4986abc4261d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:11.405466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91496bd4-682d-4426-96f7-4986abc4261d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:11.437226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1299/2565] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:11 2022] POST /identity/v3/users => generated 309 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:11.438914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0e8d3-09c2-47ed-9708-82cc1d9fbac0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:11.459609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0e8d3-09c2-47ed-9708-82cc1d9fbac0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:11.467403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abac64-39d0-4fbc-a6ee-688321bf3ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:11.468407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abac64-39d0-4fbc-a6ee-688321bf3ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:11.468979 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abac64-39d0-4fbc-a6ee-688321bf3ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:11.469452 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abac64-39d0-4fbc-a6ee-688321bf3ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:11.470321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abac64-39d0-4fbc-a6ee-688321bf3ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:11.472360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abac64-39d0-4fbc-a6ee-688321bf3ac4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:11.490617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1267/2566] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:11.491987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91496bd4-682d-4426-96f7-4986abc4261d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:11.504199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91496bd4-682d-4426-96f7-4986abc4261d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:11.511802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd79612-1e13-42fc-98c1-ecd71aa94a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:11.512891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd79612-1e13-42fc-98c1-ecd71aa94a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:11.513439 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd79612-1e13-42fc-98c1-ecd71aa94a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:11.513813 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd79612-1e13-42fc-98c1-ecd71aa94a73 admin admin] PATH_INFO: `/v3/projects/369b6fb6c3d347dfb1abe4306b6aa4e8/users/b8dce9470d914d83b23f24c95d5b72ea/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:11.532974 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd79612-1e13-42fc-98c1-ecd71aa94a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=369b6fb6c3d347dfb1abe4306b6aa4e8, user_id=b8dce9470d914d83b23f24c95d5b72ea, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:11.535783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd79612-1e13-42fc-98c1-ecd71aa94a73 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:11.555744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1300/2567] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:11 2022] PUT /identity/v3/projects/369b6fb6c3d347dfb1abe4306b6aa4e8/users/b8dce9470d914d83b23f24c95d5b72ea/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:11.557138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abac64-39d0-4fbc-a6ee-688321bf3ac4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:11.569347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abac64-39d0-4fbc-a6ee-688321bf3ac4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:11.665999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd69e16-ad6d-44dd-ad18-b78c9efad791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:11.667248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd69e16-ad6d-44dd-ad18-b78c9efad791 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:11.667892 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd69e16-ad6d-44dd-ad18-b78c9efad791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:11.668493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd69e16-ad6d-44dd-ad18-b78c9efad791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:11.669416 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd69e16-ad6d-44dd-ad18-b78c9efad791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:11.674583 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd69e16-ad6d-44dd-ad18-b78c9efad791 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:11.688068 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1268/2568] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:11.693026 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd79612-1e13-42fc-98c1-ecd71aa94a73 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:11.710399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd79612-1e13-42fc-98c1-ecd71aa94a73 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:11.716860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6d723-3499-4ec0-8b7a-c450fddf002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:11.717784 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6d723-3499-4ec0-8b7a-c450fddf002f admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:11.721860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6d723-3499-4ec0-8b7a-c450fddf002f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:11.722208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6d723-3499-4ec0-8b7a-c450fddf002f admin admin] PATH_INFO: `/v3/projects/369b6fb6c3d347dfb1abe4306b6aa4e8/users/b8dce9470d914d83b23f24c95d5b72ea/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:11.724994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6d723-3499-4ec0-8b7a-c450fddf002f admin admin] RBAC: Authorizing `identity:create_grant(project_id=369b6fb6c3d347dfb1abe4306b6aa4e8, user_id=b8dce9470d914d83b23f24c95d5b72ea, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:11.728073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6d723-3499-4ec0-8b7a-c450fddf002f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:11.741621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1301/2569] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:11 2022] PUT /identity/v3/projects/369b6fb6c3d347dfb1abe4306b6aa4e8/users/b8dce9470d914d83b23f24c95d5b72ea/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:12.507217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eecb2-4349-4084-9d94-bbdd06dee0b5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:12.507747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eecb2-4349-4084-9d94-bbdd06dee0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:12.507925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eecb2-4349-4084-9d94-bbdd06dee0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:12.517499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:12.546257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-315eecb2-4349-4084-9d94-bbdd06dee0b5 None None] MFA Rules not processed for user `b8dce9470d914d83b23f24c95d5b72ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:12.548165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-315eecb2-4349-4084-9d94-bbdd06dee0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:12.648719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1269/2570] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:14.205541 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6d723-3499-4ec0-8b7a-c450fddf002f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:14.276823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6d723-3499-4ec0-8b7a-c450fddf002f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:14.281371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:14.281997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:14.282273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:14.282543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] PATH_INFO: `/v3/users/901c78db6c0343cba1205de57c25b6d5` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:14.283647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=901c78db6c0343cba1205de57c25b6d5)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:14.285597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:14.336724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '901c78db6c0343cba1205de57c25b6d5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.343667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '901c78db6c0343cba1205de57c25b6d5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.351699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '901c78db6c0343cba1205de57c25b6d5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.352374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '901c78db6c0343cba1205de57c25b6d5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.353001 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '901c78db6c0343cba1205de57c25b6d5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.368757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:14.372899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1302/2571] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:14 2022] DELETE /identity/v3/users/901c78db6c0343cba1205de57c25b6d5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:14.562342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eecb2-4349-4084-9d94-bbdd06dee0b5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:14.573576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eecb2-4349-4084-9d94-bbdd06dee0b5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:14.655203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:14.655901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:14.656199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:14.656547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] PATH_INFO: `/v3/projects/129dde02e7f44596882782379d377a50` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:14.657690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] RBAC: Authorizing `identity:delete_project(project_id=129dde02e7f44596882782379d377a50)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:14.659668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:14.710064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129dde02e7f44596882782379d377a50', 'request_id': 'req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.715856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129dde02e7f44596882782379d377a50', 'request_id': 'req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.723147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129dde02e7f44596882782379d377a50', 'request_id': 'req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.735043 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] The token cache is being invalidate because project 129dde02e7f44596882782379d377a50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:14.735329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.735956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.737479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1270/2572] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:14 2022] DELETE /identity/v3/projects/129dde02e7f44596882782379d377a50 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:14.742230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:14.743669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:14.757887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b46bd-f17d-4033-9779-9cfa2b2137e6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:14.820062 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087b502-448d-4282-8241-492baba64621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:14.820758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087b502-448d-4282-8241-492baba64621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:14.821041 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087b502-448d-4282-8241-492baba64621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:14.821318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087b502-448d-4282-8241-492baba64621 admin admin] PATH_INFO: `/v3/users/b8dce9470d914d83b23f24c95d5b72ea` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:14.822741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087b502-448d-4282-8241-492baba64621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8dce9470d914d83b23f24c95d5b72ea)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:14.824672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087b502-448d-4282-8241-492baba64621 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:14.873272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e087b502-448d-4282-8241-492baba64621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8dce9470d914d83b23f24c95d5b72ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.881230 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e087b502-448d-4282-8241-492baba64621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8dce9470d914d83b23f24c95d5b72ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.889555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e087b502-448d-4282-8241-492baba64621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8dce9470d914d83b23f24c95d5b72ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.890186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e087b502-448d-4282-8241-492baba64621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8dce9470d914d83b23f24c95d5b72ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.890849 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e087b502-448d-4282-8241-492baba64621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8dce9470d914d83b23f24c95d5b72ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:14.908661 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1303/2573] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:14 2022] DELETE /identity/v3/users/b8dce9470d914d83b23f24c95d5b72ea => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:15.124654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:15.126504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:15.142034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe4577-c229-4dcb-87a6-e4c60fbb74da admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:15.156751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:15.215161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:15.215690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.215971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.216240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] PATH_INFO: `/v3/projects/369b6fb6c3d347dfb1abe4306b6aa4e8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.217390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] RBAC: Authorizing `identity:delete_project(project_id=369b6fb6c3d347dfb1abe4306b6aa4e8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:15.219272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:15.268328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369b6fb6c3d347dfb1abe4306b6aa4e8', 'request_id': 'req-4c0dd457-2df9-434d-bb48-aa792275f05a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:15.276496 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369b6fb6c3d347dfb1abe4306b6aa4e8', 'request_id': 'req-4c0dd457-2df9-434d-bb48-aa792275f05a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:15.290719 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369b6fb6c3d347dfb1abe4306b6aa4e8', 'request_id': 'req-4c0dd457-2df9-434d-bb48-aa792275f05a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:15.306466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] The token cache is being invalidate because project 369b6fb6c3d347dfb1abe4306b6aa4e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:15.306902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:15.307733 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4c0dd457-2df9-434d-bb48-aa792275f05a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:15.310045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1271/2574] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:15 2022] DELETE /identity/v3/projects/369b6fb6c3d347dfb1abe4306b6aa4e8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:15.314971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e6e50-22a8-4f5e-ac7e-ec5bd1f3ceda None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.315595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e6e50-22a8-4f5e-ac7e-ec5bd1f3ceda None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.315806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e6e50-22a8-4f5e-ac7e-ec5bd1f3ceda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.337020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b68e6e50-22a8-4f5e-ac7e-ec5bd1f3ceda None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:15.338516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b68e6e50-22a8-4f5e-ac7e-ec5bd1f3ceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:15.402402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1304/2575] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:15.412132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbab18-542f-4198-8af1-917869630519 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.412416 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbab18-542f-4198-8af1-917869630519 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.412730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbab18-542f-4198-8af1-917869630519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.434352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-75bbab18-542f-4198-8af1-917869630519 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:15.435587 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-75bbab18-542f-4198-8af1-917869630519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:15.443625 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1272/2576] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:15.447977 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e6e50-22a8-4f5e-ac7e-ec5bd1f3ceda None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:15.457480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e6e50-22a8-4f5e-ac7e-ec5bd1f3ceda None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:15.462801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:15.463560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.463835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.464105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.465333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:15.465333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:15.466820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:15.480006 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1305/2577] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:21:15.484367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbab18-542f-4198-8af1-917869630519 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:15.495028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbab18-542f-4198-8af1-917869630519 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:15.499885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d8b12-5c41-402e-9097-6edcdde3ad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:15.500538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8b12-5c41-402e-9097-6edcdde3ad6d admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.500846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8b12-5c41-402e-9097-6edcdde3ad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.501131 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8b12-5c41-402e-9097-6edcdde3ad6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.501813 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d8b12-5c41-402e-9097-6edcdde3ad6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:15.503757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d8b12-5c41-402e-9097-6edcdde3ad6d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:15.531698 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1273/2578] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:15 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:15.536743 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:15.542319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:15.549139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da1827-a73b-4859-982c-9ce968af3750 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:15.609513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1b05d-e517-4bd6-adfb-f4cbe32af572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:15.610122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1b05d-e517-4bd6-adfb-f4cbe32af572 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.610400 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1b05d-e517-4bd6-adfb-f4cbe32af572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.610673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1b05d-e517-4bd6-adfb-f4cbe32af572 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.611299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1b05d-e517-4bd6-adfb-f4cbe32af572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:15.613318 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1b05d-e517-4bd6-adfb-f4cbe32af572 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:15.638998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1306/2579] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:15 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:15.644582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d8b12-5c41-402e-9097-6edcdde3ad6d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:15.659354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d8b12-5c41-402e-9097-6edcdde3ad6d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:15.665433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35abb9-28e2-4c3e-970f-e0342ef56e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:15.666080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35abb9-28e2-4c3e-970f-e0342ef56e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.666361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35abb9-28e2-4c3e-970f-e0342ef56e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.666633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35abb9-28e2-4c3e-970f-e0342ef56e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.667295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35abb9-28e2-4c3e-970f-e0342ef56e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:15.668994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35abb9-28e2-4c3e-970f-e0342ef56e8c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:15.684546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1274/2580] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:15.685799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1b05d-e517-4bd6-adfb-f4cbe32af572 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:15.697816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1b05d-e517-4bd6-adfb-f4cbe32af572 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:15.702559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea080a9c-c72e-4837-ac24-d14e193cd754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:15.703184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080a9c-c72e-4837-ac24-d14e193cd754 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.703460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080a9c-c72e-4837-ac24-d14e193cd754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.703739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea080a9c-c72e-4837-ac24-d14e193cd754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.713349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea080a9c-c72e-4837-ac24-d14e193cd754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:15.713349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea080a9c-c72e-4837-ac24-d14e193cd754 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:15.722040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1307/2581] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:15.726847 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35abb9-28e2-4c3e-970f-e0342ef56e8c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:15.736861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35abb9-28e2-4c3e-970f-e0342ef56e8c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:15.741748 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188214c-e854-4c2c-a2be-e26a6de357e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:15.742410 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188214c-e854-4c2c-a2be-e26a6de357e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.742760 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188214c-e854-4c2c-a2be-e26a6de357e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.742936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188214c-e854-4c2c-a2be-e26a6de357e7 admin admin] PATH_INFO: `/v3/projects/fd66e5f52dbf47df9e6dd74d073e7ca8/users/2ebb407ccb8e4627ad37e66d9cb5bc01/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.764536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188214c-e854-4c2c-a2be-e26a6de357e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd66e5f52dbf47df9e6dd74d073e7ca8, user_id=2ebb407ccb8e4627ad37e66d9cb5bc01, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:15.767159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188214c-e854-4c2c-a2be-e26a6de357e7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:15.784851 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1275/2582] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:15 2022] PUT /identity/v3/projects/fd66e5f52dbf47df9e6dd74d073e7ca8/users/2ebb407ccb8e4627ad37e66d9cb5bc01/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:15.796382 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c457e-786f-40cd-8227-0fca7988944a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.796684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c457e-786f-40cd-8227-0fca7988944a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.796951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c457e-786f-40cd-8227-0fca7988944a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.829490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-180c457e-786f-40cd-8227-0fca7988944a None None] MFA Rules not processed for user `2ebb407ccb8e4627ad37e66d9cb5bc01`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:15.830754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-180c457e-786f-40cd-8227-0fca7988944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:15.907709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1308/2583] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:15.913123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188214c-e854-4c2c-a2be-e26a6de357e7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:15.967288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:15.991615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188214c-e854-4c2c-a2be-e26a6de357e7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:15.996652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e63e6c-13b2-4364-9db9-9b4b347eecba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:15.997278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e63e6c-13b2-4364-9db9-9b4b347eecba admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:15.997559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e63e6c-13b2-4364-9db9-9b4b347eecba admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:15.997837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e63e6c-13b2-4364-9db9-9b4b347eecba admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:15.998462 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e63e6c-13b2-4364-9db9-9b4b347eecba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:16.000536 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e63e6c-13b2-4364-9db9-9b4b347eecba admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:16.032524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1276/2584] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:15 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:16.037498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c457e-786f-40cd-8227-0fca7988944a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:16.052425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c457e-786f-40cd-8227-0fca7988944a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:16.110216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d696f4-caed-4881-bf77-bf93f6f76174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:16.110987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d696f4-caed-4881-bf77-bf93f6f76174 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:16.111264 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d696f4-caed-4881-bf77-bf93f6f76174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:16.111536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d696f4-caed-4881-bf77-bf93f6f76174 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:16.112304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d696f4-caed-4881-bf77-bf93f6f76174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:16.115457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d696f4-caed-4881-bf77-bf93f6f76174 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:16.131712 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:16.137490 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1309/2585] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:16 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:16.141928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e63e6c-13b2-4364-9db9-9b4b347eecba admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:16.153320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e63e6c-13b2-4364-9db9-9b4b347eecba admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:16.158254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49734875-7313-45ed-ac5b-bd4aca1590be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:16.158878 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49734875-7313-45ed-ac5b-bd4aca1590be admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:16.159154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49734875-7313-45ed-ac5b-bd4aca1590be admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:16.159428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49734875-7313-45ed-ac5b-bd4aca1590be admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:16.160106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49734875-7313-45ed-ac5b-bd4aca1590be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:16.162657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49734875-7313-45ed-ac5b-bd4aca1590be admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:16.174910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1277/2586] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:16.179983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d696f4-caed-4881-bf77-bf93f6f76174 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:16.190108 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d696f4-caed-4881-bf77-bf93f6f76174 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:16.195747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244b54-7939-4455-a94d-eee9f69eee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:16.196422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244b54-7939-4455-a94d-eee9f69eee63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:16.196735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244b54-7939-4455-a94d-eee9f69eee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:16.197015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe244b54-7939-4455-a94d-eee9f69eee63 admin admin] PATH_INFO: `/v3/projects/2899931293244e7fb885612fd99478aa/users/104657520b574a94ad67d020abaa78b7/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:16.213807 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe244b54-7939-4455-a94d-eee9f69eee63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2899931293244e7fb885612fd99478aa, user_id=104657520b574a94ad67d020abaa78b7, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:16.217250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe244b54-7939-4455-a94d-eee9f69eee63 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:16.230761 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1310/2587] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:16 2022] PUT /identity/v3/projects/2899931293244e7fb885612fd99478aa/users/104657520b574a94ad67d020abaa78b7/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:16.234608 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49734875-7313-45ed-ac5b-bd4aca1590be admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:16.245145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49734875-7313-45ed-ac5b-bd4aca1590be admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:16.303111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d4b60-9e59-4231-a829-e6770ea4c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:16.303599 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d4b60-9e59-4231-a829-e6770ea4c0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:16.303879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d4b60-9e59-4231-a829-e6770ea4c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:16.304151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d4b60-9e59-4231-a829-e6770ea4c0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:16.304827 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d4b60-9e59-4231-a829-e6770ea4c0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:16.306423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d4b60-9e59-4231-a829-e6770ea4c0d0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:16.316732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1278/2588] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:16.321728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244b54-7939-4455-a94d-eee9f69eee63 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:16.332273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe244b54-7939-4455-a94d-eee9f69eee63 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:16.338332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:16.339010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:16.339291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:16.339566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] PATH_INFO: `/v3/projects/2899931293244e7fb885612fd99478aa/users/104657520b574a94ad67d020abaa78b7/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:16.341530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2899931293244e7fb885612fd99478aa, user_id=104657520b574a94ad67d020abaa78b7, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:16.344168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:16.354891 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1311/2589] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:16 2022] PUT /identity/v3/projects/2899931293244e7fb885612fd99478aa/users/104657520b574a94ad67d020abaa78b7/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:16.365621 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb774a-9dae-40e2-8333-c85f37a22295 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:16.365903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb774a-9dae-40e2-8333-c85f37a22295 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:16.366178 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb774a-9dae-40e2-8333-c85f37a22295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:16.395305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a6fb774a-9dae-40e2-8333-c85f37a22295 None None] MFA Rules not processed for user `104657520b574a94ad67d020abaa78b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:16.396568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a6fb774a-9dae-40e2-8333-c85f37a22295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:16.468732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1279/2590] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:16.479962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:16.481782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:16.541792 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b1fca-b204-4f21-8ea3-7e1f4ff24ecb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:16.545791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd51a0-94b5-4906-9648-83cf00290a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:16.546394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd51a0-94b5-4906-9648-83cf00290a71 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:16.546674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd51a0-94b5-4906-9648-83cf00290a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:16.546950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd51a0-94b5-4906-9648-83cf00290a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:16.556973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd51a0-94b5-4906-9648-83cf00290a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:16.558407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd51a0-94b5-4906-9648-83cf00290a71 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:16.569610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1312/2591] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:18.223069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb774a-9dae-40e2-8333-c85f37a22295 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:18.268678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:18.289944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb774a-9dae-40e2-8333-c85f37a22295 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:18.295270 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:18.295900 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:18.296175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:18.296463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] PATH_INFO: `/v3/users/2ebb407ccb8e4627ad37e66d9cb5bc01` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:18.297586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ebb407ccb8e4627ad37e66d9cb5bc01)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:18.299486 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:18.349166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ebb407ccb8e4627ad37e66d9cb5bc01'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.356883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ebb407ccb8e4627ad37e66d9cb5bc01'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.363829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ebb407ccb8e4627ad37e66d9cb5bc01'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.364784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ebb407ccb8e4627ad37e66d9cb5bc01'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.365470 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ebb407ccb8e4627ad37e66d9cb5bc01'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.384480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1280/2592] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:18 2022] DELETE /identity/v3/users/2ebb407ccb8e4627ad37e66d9cb5bc01 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:18.388952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a24e68-0cae-4bba-8f2e-d59257f51d56 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:18.389404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a24e68-0cae-4bba-8f2e-d59257f51d56 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:18.389500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a24e68-0cae-4bba-8f2e-d59257f51d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:18.408994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-24a24e68-0cae-4bba-8f2e-d59257f51d56 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:18.410189 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-24a24e68-0cae-4bba-8f2e-d59257f51d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:18.473373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1313/2593] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:18.483740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:18.485246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:18.550223 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d027-099c-4603-95ea-20f6cb23006c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:18.554186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:18.554798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:18.555082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:18.555356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:18.565052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:18.568508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:18.578390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1281/2594] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:18.829510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a24e68-0cae-4bba-8f2e-d59257f51d56 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:18.831077 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-24a24e68-0cae-4bba-8f2e-d59257f51d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:18.844403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a24e68-0cae-4bba-8f2e-d59257f51d56 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:18.849966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:18.850839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:18.851111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:18.851381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] PATH_INFO: `/v3/users/5f58617086df4c62a4593ee61ae26409` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:18.852533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f58617086df4c62a4593ee61ae26409)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:18.854968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:18.880577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:18.883174 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:18.906303 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d1a9e-7ae7-4e81-b9e4-eb40fc437db4 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:18.912658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:18.913304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:18.913644 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:18.914123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] PATH_INFO: `/v3/projects/fd66e5f52dbf47df9e6dd74d073e7ca8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:18.915275 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd66e5f52dbf47df9e6dd74d073e7ca8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:18.916185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f58617086df4c62a4593ee61ae26409'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.917527 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:18.925320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f58617086df4c62a4593ee61ae26409'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.929087 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:18.933865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f58617086df4c62a4593ee61ae26409'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.934987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f58617086df4c62a4593ee61ae26409'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.935731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1c607c92-f569-4f43-86ee-e3f348b10277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f58617086df4c62a4593ee61ae26409'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.965181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1314/2595] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:18 2022] DELETE /identity/v3/users/5f58617086df4c62a4593ee61ae26409 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:18.966961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7574-4987-4d29-9046-62f602ae3532 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:18.967254 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7574-4987-4d29-9046-62f602ae3532 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:18.967535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7574-4987-4d29-9046-62f602ae3532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:18.978775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd66e5f52dbf47df9e6dd74d073e7ca8', 'request_id': 'req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.983773 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:18.986003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd66e5f52dbf47df9e6dd74d073e7ca8', 'request_id': 'req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:18.987405 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-692b7574-4987-4d29-9046-62f602ae3532 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:18.988670 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-692b7574-4987-4d29-9046-62f602ae3532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:18.996176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd66e5f52dbf47df9e6dd74d073e7ca8', 'request_id': 'req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.010194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] The token cache is being invalidate because project fd66e5f52dbf47df9e6dd74d073e7ca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:19.010481 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.011139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.012689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1282/2596] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:18 2022] DELETE /identity/v3/projects/fd66e5f52dbf47df9e6dd74d073e7ca8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:19.016479 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:19.017787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:19.058519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1315/2597] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:19.069458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b7574-4987-4d29-9046-62f602ae3532 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:19.070855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-692b7574-4987-4d29-9046-62f602ae3532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:19.090287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bf323-ae11-40c9-ac03-b9cfad2635cd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:19.094992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:19.095624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.095902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.096175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] PATH_INFO: `/v3/users/104657520b574a94ad67d020abaa78b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.097322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=104657520b574a94ad67d020abaa78b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:19.099650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:19.131458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b7574-4987-4d29-9046-62f602ae3532 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:19.135485 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:19.136161 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.140493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.140493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.148335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:19.149876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:19.159263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '104657520b574a94ad67d020abaa78b7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.160756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1316/2598] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:19.171207 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '104657520b574a94ad67d020abaa78b7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.181052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '104657520b574a94ad67d020abaa78b7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.181052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '104657520b574a94ad67d020abaa78b7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.181052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '104657520b574a94ad67d020abaa78b7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.201843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1283/2599] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:19 2022] DELETE /identity/v3/users/104657520b574a94ad67d020abaa78b7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:19.562931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:19.564585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:19.579285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c44ac-9fc8-4349-98bb-f180cc820d87 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:19.587115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:19.590003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:19.653483 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:19.654219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.654566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.654909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] PATH_INFO: `/v3/projects/b5cf841803c3483ba23a28309b25a8d5` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.656153 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5cf841803c3483ba23a28309b25a8d5)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:19.658279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:19.670761 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26816de-365b-4537-9e8b-a30435570b99 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:19.675394 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:19.676017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.676300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.676633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] PATH_INFO: `/v3/projects/2899931293244e7fb885612fd99478aa` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.677746 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2899931293244e7fb885612fd99478aa)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:19.679627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:19.685959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:19.689059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:19.714690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5cf841803c3483ba23a28309b25a8d5', 'request_id': 'req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.720508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5cf841803c3483ba23a28309b25a8d5', 'request_id': 'req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.728834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5cf841803c3483ba23a28309b25a8d5', 'request_id': 'req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.735720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2899931293244e7fb885612fd99478aa', 'request_id': 'req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.741816 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2899931293244e7fb885612fd99478aa', 'request_id': 'req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.745709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] The token cache is being invalidate because project b5cf841803c3483ba23a28309b25a8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:19.746105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.746856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.748600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1317/2600] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:19 2022] DELETE /identity/v3/projects/b5cf841803c3483ba23a28309b25a8d5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:19.749877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2899931293244e7fb885612fd99478aa', 'request_id': 'req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.752591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:19.754011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:19.762588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] The token cache is being invalidate because project 2899931293244e7fb885612fd99478aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:19.762883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.763542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ac6e68d1-71b0-4b44-b56a-0b0498930ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.765150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1284/2601] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:19 2022] DELETE /identity/v3/projects/2899931293244e7fb885612fd99478aa => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:19.771994 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc88cb-82d4-4837-9eb7-9609321c7fc1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.772299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc88cb-82d4-4837-9eb7-9609321c7fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.773126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc88cb-82d4-4837-9eb7-9609321c7fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.794474 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5dfc88cb-82d4-4837-9eb7-9609321c7fc1 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:19.795987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5dfc88cb-82d4-4837-9eb7-9609321c7fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:19.824284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4d71d-2ec1-426d-b9dd-f22e82eae792 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:19.865524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1285/2602] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:19.874523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e60e49-7480-4e1c-a746-cbbc110ec6d1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.874835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e60e49-7480-4e1c-a746-cbbc110ec6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.875081 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e60e49-7480-4e1c-a746-cbbc110ec6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.886996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:19.887651 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.887926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.888192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] PATH_INFO: `/v3/users/6fabe84e22354cd7848d54046d862cb1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.889457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fabe84e22354cd7848d54046d862cb1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:19.891406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:19.897196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b7e60e49-7480-4e1c-a746-cbbc110ec6d1 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:19.898381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b7e60e49-7480-4e1c-a746-cbbc110ec6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:19.908045 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1286/2603] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:19.912564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e60e49-7480-4e1c-a746-cbbc110ec6d1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:19.922335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e60e49-7480-4e1c-a746-cbbc110ec6d1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:19.927516 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:19.928114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.928388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.928818 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.929473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:19.929776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:19.931387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:19.944382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1287/2604] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:21:19.944778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fabe84e22354cd7848d54046d862cb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.948871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:19.954363 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fabe84e22354cd7848d54046d862cb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.958709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:19.960649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04386cc4-e5aa-4571-bab2-2fbaec65d22f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:19.963679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fabe84e22354cd7848d54046d862cb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.964708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fabe84e22354cd7848d54046d862cb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.966013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fabe84e22354cd7848d54046d862cb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:19.966658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841a1e6-5091-4588-a813-ffe4e024394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:19.967257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a1e6-5091-4588-a813-ffe4e024394f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:19.967535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a1e6-5091-4588-a813-ffe4e024394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:19.967835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a1e6-5091-4588-a813-ffe4e024394f admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:19.968525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841a1e6-5091-4588-a813-ffe4e024394f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:19.970445 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841a1e6-5091-4588-a813-ffe4e024394f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:19.987444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1318/2605] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:19 2022] DELETE /identity/v3/users/6fabe84e22354cd7848d54046d862cb1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:19.999736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1288/2606] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:19 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:20.003904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.005989 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.078733 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eff25b-2f5e-4ff7-9d14-4862c0e56f83 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.140868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45df37-699c-493b-a088-ea2f6423b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.141393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45df37-699c-493b-a088-ea2f6423b2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.141675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45df37-699c-493b-a088-ea2f6423b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.141951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45df37-699c-493b-a088-ea2f6423b2a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.142589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45df37-699c-493b-a088-ea2f6423b2a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.144867 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45df37-699c-493b-a088-ea2f6423b2a1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.175010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1319/2607] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:20 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:20.179572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841a1e6-5091-4588-a813-ffe4e024394f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.190096 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841a1e6-5091-4588-a813-ffe4e024394f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.195737 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da535a6-d7cb-45bb-82cb-4684cc8ab3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.196423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da535a6-d7cb-45bb-82cb-4684cc8ab3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.196735 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da535a6-d7cb-45bb-82cb-4684cc8ab3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.197054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da535a6-d7cb-45bb-82cb-4684cc8ab3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.197853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da535a6-d7cb-45bb-82cb-4684cc8ab3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.199687 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da535a6-d7cb-45bb-82cb-4684cc8ab3b1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.216499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1289/2608] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:20.218144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45df37-699c-493b-a088-ea2f6423b2a1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.230361 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45df37-699c-493b-a088-ea2f6423b2a1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.238276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.238276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.238276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.238276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.238734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.240139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.251095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1320/2609] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:20.255929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da535a6-d7cb-45bb-82cb-4684cc8ab3b1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.266304 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da535a6-d7cb-45bb-82cb-4684cc8ab3b1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.275565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cb6b2-e9eb-4eaf-a4dc-12a530304ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.276261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cb6b2-e9eb-4eaf-a4dc-12a530304ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.276597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cb6b2-e9eb-4eaf-a4dc-12a530304ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.276859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cb6b2-e9eb-4eaf-a4dc-12a530304ad5 admin admin] PATH_INFO: `/v3/projects/6dfcda1b86904dcebaad107ec31e37de/users/9720b171ade04736992148332dc2c2c9/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.294860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.297048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.303491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cb6b2-e9eb-4eaf-a4dc-12a530304ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dfcda1b86904dcebaad107ec31e37de, user_id=9720b171ade04736992148332dc2c2c9, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.307435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cb6b2-e9eb-4eaf-a4dc-12a530304ad5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.313961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bcb95-a1a3-4bc3-bc5a-a336bf4e309b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.322015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.323368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.323951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.324197 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] PATH_INFO: `/v3/projects/e8b03732e5ad479d859a7de3fd59aee5` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.326290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8b03732e5ad479d859a7de3fd59aee5)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.328375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.341418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1290/2610] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:20 2022] PUT /identity/v3/projects/6dfcda1b86904dcebaad107ec31e37de/users/9720b171ade04736992148332dc2c2c9/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:20.351810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3237-2258-4baf-a634-22e2a872d4ce None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.352090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3237-2258-4baf-a634-22e2a872d4ce None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.352387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3237-2258-4baf-a634-22e2a872d4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.389956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a95b3237-2258-4baf-a634-22e2a872d4ce None None] MFA Rules not processed for user `9720b171ade04736992148332dc2c2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:20.391307 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a95b3237-2258-4baf-a634-22e2a872d4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.398295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:20.398426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8b03732e5ad479d859a7de3fd59aee5', 'request_id': 'req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.407322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8b03732e5ad479d859a7de3fd59aee5', 'request_id': 'req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.409496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:20.417499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8b03732e5ad479d859a7de3fd59aee5', 'request_id': 'req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.431729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] The token cache is being invalidate because project e8b03732e5ad479d859a7de3fd59aee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:20.432054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.432774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dda78b55-c2ef-4c70-a4d7-2e2950dfaeb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.434856 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1321/2611] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:20 2022] DELETE /identity/v3/projects/e8b03732e5ad479d859a7de3fd59aee5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:20.439853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d266c8-6cdf-48f1-9963-328851802059 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.440139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d266c8-6cdf-48f1-9963-328851802059 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.440408 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d266c8-6cdf-48f1-9963-328851802059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.460875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-53d266c8-6cdf-48f1-9963-328851802059 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:20.462079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-53d266c8-6cdf-48f1-9963-328851802059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.466467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1291/2612] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:20.479500 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3237-2258-4baf-a634-22e2a872d4ce None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.480903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a95b3237-2258-4baf-a634-22e2a872d4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.528666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1322/2613] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:20.536750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2877d1-86c3-42ab-810c-0952b93d2946 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.537045 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2877d1-86c3-42ab-810c-0952b93d2946 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.537330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2877d1-86c3-42ab-810c-0952b93d2946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.543313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3237-2258-4baf-a634-22e2a872d4ce None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.547214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.547960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.548311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.548627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.558751 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.559289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8a2877d1-86c3-42ab-810c-0952b93d2946 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:20.560213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.560512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8a2877d1-86c3-42ab-810c-0952b93d2946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.568582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1323/2614] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:20.569767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1292/2615] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:20.572655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2877d1-86c3-42ab-810c-0952b93d2946 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.581868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2877d1-86c3-42ab-810c-0952b93d2946 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.589270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.589960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.590237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.590872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.591565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:20.592119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.592945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.593808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.594339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.607652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1324/2616] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:21:20.609478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9da53f-e49e-4e3a-b593-cdc5f1b69939 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.611842 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.615215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.615902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.616177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.616457 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] PATH_INFO: `/v3/users/9720b171ade04736992148332dc2c2c9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.617741 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9720b171ade04736992148332dc2c2c9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.619967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.623669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c8874-c526-42f5-ac46-7a311c204951 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.628469 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e16f91-b512-4de2-b63a-c26cee691b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.629096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e16f91-b512-4de2-b63a-c26cee691b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.629403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e16f91-b512-4de2-b63a-c26cee691b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.629764 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e16f91-b512-4de2-b63a-c26cee691b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.630421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e16f91-b512-4de2-b63a-c26cee691b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.632349 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e16f91-b512-4de2-b63a-c26cee691b07 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.662694 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1325/2617] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:20 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:20.666658 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e16f91-b512-4de2-b63a-c26cee691b07 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.675559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9720b171ade04736992148332dc2c2c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.685044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9720b171ade04736992148332dc2c2c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.694769 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9720b171ade04736992148332dc2c2c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.695511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9720b171ade04736992148332dc2c2c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.696237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464e13b3-81bb-465c-b530-44b081ec2184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9720b171ade04736992148332dc2c2c9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:20.715029 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1293/2618] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:20 2022] DELETE /identity/v3/users/9720b171ade04736992148332dc2c2c9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:20.719619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6e32e-3c37-4cf9-a1c7-e51619333d00 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.720136 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6e32e-3c37-4cf9-a1c7-e51619333d00 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.720412 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6e32e-3c37-4cf9-a1c7-e51619333d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.740347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e16f91-b512-4de2-b63a-c26cee691b07 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.741168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8aa6e32e-3c37-4cf9-a1c7-e51619333d00 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:20.742399 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8aa6e32e-3c37-4cf9-a1c7-e51619333d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.760312 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:20.809396 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1294/2619] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:20.819834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6e32e-3c37-4cf9-a1c7-e51619333d00 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.821362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8aa6e32e-3c37-4cf9-a1c7-e51619333d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.887911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6e32e-3c37-4cf9-a1c7-e51619333d00 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.894982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.894982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.894982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.894982 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.904504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.905944 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.916547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1295/2620] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:20.930285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:20.945130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e87ae5-b4b4-412f-923a-45f382b71c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.945755 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87ae5-b4b4-412f-923a-45f382b71c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:20.946040 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87ae5-b4b4-412f-923a-45f382b71c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:20.946331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87ae5-b4b4-412f-923a-45f382b71c14 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:20.946992 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e87ae5-b4b4-412f-923a-45f382b71c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:20.948966 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e87ae5-b4b4-412f-923a-45f382b71c14 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:20.972510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1326/2621] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:20 2022] POST /identity/v3/users => generated 284 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:20.976852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:20.978364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:20.994216 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cf323-a804-4a66-80e2-d709e28f0568 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:20.999062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d7cc9-d253-4f59-a772-8898d20b1e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:20.999688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d7cc9-d253-4f59-a772-8898d20b1e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.000150 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d7cc9-d253-4f59-a772-8898d20b1e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.000482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d7cc9-d253-4f59-a772-8898d20b1e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.001163 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d7cc9-d253-4f59-a772-8898d20b1e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.002757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d7cc9-d253-4f59-a772-8898d20b1e6c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.016053 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1296/2622] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:21.019999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e87ae5-b4b4-412f-923a-45f382b71c14 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.031701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e87ae5-b4b4-412f-923a-45f382b71c14 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.037283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271a66b-1756-4716-89ff-250a128c352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.037875 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a66b-1756-4716-89ff-250a128c352e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.038150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a66b-1756-4716-89ff-250a128c352e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.038418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a66b-1756-4716-89ff-250a128c352e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.039116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271a66b-1756-4716-89ff-250a128c352e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.040734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271a66b-1756-4716-89ff-250a128c352e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.056212 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1327/2623] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:21.057628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d7cc9-d253-4f59-a772-8898d20b1e6c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.069064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d7cc9-d253-4f59-a772-8898d20b1e6c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.074218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.074886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.075248 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.075553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] PATH_INFO: `/v3/projects/b2bd6110dec1492e92cd550eb0071afb/users/ef26a12440d840068314b36d5cdc1f21/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.100084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2bd6110dec1492e92cd550eb0071afb, user_id=ef26a12440d840068314b36d5cdc1f21, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.102935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.117311 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1297/2624] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:21 2022] PUT /identity/v3/projects/b2bd6110dec1492e92cd550eb0071afb/users/ef26a12440d840068314b36d5cdc1f21/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:21.128991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0966-b2f4-434a-a582-ff0c8db6648b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.130234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0966-b2f4-434a-a582-ff0c8db6648b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.130519 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd0966-b2f4-434a-a582-ff0c8db6648b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.159948 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-99cd0966-b2f4-434a-a582-ff0c8db6648b None None] MFA Rules not processed for user `ef26a12440d840068314b36d5cdc1f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:21.162471 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-99cd0966-b2f4-434a-a582-ff0c8db6648b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:21.218565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.220543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:21.241605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1328/2625] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:21.245798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd0966-b2f4-434a-a582-ff0c8db6648b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.305434 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c5bd3-ca12-4ba3-830c-84e382509f7b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.310359 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.310987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.311257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.311545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] PATH_INFO: `/v3/projects/6dfcda1b86904dcebaad107ec31e37de` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.312795 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dfcda1b86904dcebaad107ec31e37de)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.314877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.319713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd0966-b2f4-434a-a582-ff0c8db6648b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.325888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7a87d-39d3-4500-a339-4d6a0b410c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.326496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7a87d-39d3-4500-a339-4d6a0b410c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.326770 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7a87d-39d3-4500-a339-4d6a0b410c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.327038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7a87d-39d3-4500-a339-4d6a0b410c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.327663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7a87d-39d3-4500-a339-4d6a0b410c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.329595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7a87d-39d3-4500-a339-4d6a0b410c26 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.353020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:21.360233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1329/2626] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:21.363566 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7a87d-39d3-4500-a339-4d6a0b410c26 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.372024 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dfcda1b86904dcebaad107ec31e37de', 'request_id': 'req-4130be29-c39a-4c20-843e-a737f634e7fe'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:21.377591 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dfcda1b86904dcebaad107ec31e37de', 'request_id': 'req-4130be29-c39a-4c20-843e-a737f634e7fe'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:21.389229 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dfcda1b86904dcebaad107ec31e37de', 'request_id': 'req-4130be29-c39a-4c20-843e-a737f634e7fe'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:21.417476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] The token cache is being invalidate because project 6dfcda1b86904dcebaad107ec31e37de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:21.417989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:21.418985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4130be29-c39a-4c20-843e-a737f634e7fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:21.421709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1298/2627] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:21 2022] DELETE /identity/v3/projects/6dfcda1b86904dcebaad107ec31e37de => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:21.432418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d95d2f-f14e-479d-b553-49f9cda66207 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.432740 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d95d2f-f14e-479d-b553-49f9cda66207 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.433057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d95d2f-f14e-479d-b553-49f9cda66207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.446907 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:21.457991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b8d95d2f-f14e-479d-b553-49f9cda66207 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:21.459208 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b8d95d2f-f14e-479d-b553-49f9cda66207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:21.466734 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7a87d-39d3-4500-a339-4d6a0b410c26 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.469113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1299/2628] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:21.471656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.472696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.472696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.473205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.474798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.476939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.478132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8586371-76a0-44e6-a5ad-dff62380b703 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.478416 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8586371-76a0-44e6-a5ad-dff62380b703 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.478689 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8586371-76a0-44e6-a5ad-dff62380b703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.500264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b8586371-76a0-44e6-a5ad-dff62380b703 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:21.501492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b8586371-76a0-44e6-a5ad-dff62380b703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:21.503357 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1330/2629] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:21.507902 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.509609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:21.511727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1300/2630] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:21.515571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8586371-76a0-44e6-a5ad-dff62380b703 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.523110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f8dd0-f46a-4c40-bde7-f1e403aaa80e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.525501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8586371-76a0-44e6-a5ad-dff62380b703 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.528387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8122898-c6a8-4c08-b397-e87bcfc1529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.529016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8122898-c6a8-4c08-b397-e87bcfc1529e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.529578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8122898-c6a8-4c08-b397-e87bcfc1529e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.529578 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8122898-c6a8-4c08-b397-e87bcfc1529e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.530108 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.530240 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8122898-c6a8-4c08-b397-e87bcfc1529e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.530815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.531190 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.531434 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.532002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8122898-c6a8-4c08-b397-e87bcfc1529e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.532125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:21.532360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.534125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.543860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1331/2631] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:21:21.548621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8122898-c6a8-4c08-b397-e87bcfc1529e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.551117 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1301/2632] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:21:21.555239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.561821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8122898-c6a8-4c08-b397-e87bcfc1529e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.565146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727764b-b63a-4473-a021-bdcac7cc9553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.566399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727764b-b63a-4473-a021-bdcac7cc9553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.566399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727764b-b63a-4473-a021-bdcac7cc9553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.566399 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727764b-b63a-4473-a021-bdcac7cc9553 admin admin] PATH_INFO: `/v3/projects/3c0cddb295c349f3b2a0ae4f513dbeca/users/de7fac78536d41c0a96b3c64e3a7b4fa/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.571077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63b4c-ca26-4478-9276-8a9db8acc453 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.576063 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c6c12-b7eb-4cce-a4da-877e5a1837f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.576677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c6c12-b7eb-4cce-a4da-877e5a1837f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.576952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c6c12-b7eb-4cce-a4da-877e5a1837f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.577323 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c6c12-b7eb-4cce-a4da-877e5a1837f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.577858 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c6c12-b7eb-4cce-a4da-877e5a1837f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.579763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c6c12-b7eb-4cce-a4da-877e5a1837f9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.582378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727764b-b63a-4473-a021-bdcac7cc9553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0cddb295c349f3b2a0ae4f513dbeca, user_id=de7fac78536d41c0a96b3c64e3a7b4fa, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.586467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727764b-b63a-4473-a021-bdcac7cc9553 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.602130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1332/2633] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:21 2022] PUT /identity/v3/projects/3c0cddb295c349f3b2a0ae4f513dbeca/users/de7fac78536d41c0a96b3c64e3a7b4fa/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:21.606431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727764b-b63a-4473-a021-bdcac7cc9553 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.611105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1302/2634] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:21.615928 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c6c12-b7eb-4cce-a4da-877e5a1837f9 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.678579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727764b-b63a-4473-a021-bdcac7cc9553 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.693503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c6c12-b7eb-4cce-a4da-877e5a1837f9 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.698960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a35fa-75ca-4057-998c-70c7162b40ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.699582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a35fa-75ca-4057-998c-70c7162b40ea admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.699865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a35fa-75ca-4057-998c-70c7162b40ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.700138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a35fa-75ca-4057-998c-70c7162b40ea admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.701393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a35fa-75ca-4057-998c-70c7162b40ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.703953 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a35fa-75ca-4057-998c-70c7162b40ea admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.729651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1303/2635] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:21.733588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a35fa-75ca-4057-998c-70c7162b40ea admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.743967 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a35fa-75ca-4057-998c-70c7162b40ea admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.748891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee7332-e229-4838-9ecd-4431321ec4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.749511 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee7332-e229-4838-9ecd-4431321ec4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.749788 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee7332-e229-4838-9ecd-4431321ec4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.750065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee7332-e229-4838-9ecd-4431321ec4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.750713 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee7332-e229-4838-9ecd-4431321ec4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.752294 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee7332-e229-4838-9ecd-4431321ec4b5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.752487 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb46fe-025f-475a-88de-3e347481f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.753270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb46fe-025f-475a-88de-3e347481f7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.753703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb46fe-025f-475a-88de-3e347481f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.754042 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb46fe-025f-475a-88de-3e347481f7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.756141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb46fe-025f-475a-88de-3e347481f7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.757505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb46fe-025f-475a-88de-3e347481f7c1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.768015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1304/2636] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:21:21.769411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee7332-e229-4838-9ecd-4431321ec4b5 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.772378 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1333/2637] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:21.776488 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb46fe-025f-475a-88de-3e347481f7c1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.786165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee7332-e229-4838-9ecd-4431321ec4b5 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.786974 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb46fe-025f-475a-88de-3e347481f7c1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.791326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f61b94-01c2-4a04-9865-13bf28280055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.791923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ea70e-be4d-46bc-b329-6834fa2ccc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.792139 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f61b94-01c2-4a04-9865-13bf28280055 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.792332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f61b94-01c2-4a04-9865-13bf28280055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.792647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f61b94-01c2-4a04-9865-13bf28280055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.792972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ea70e-be4d-46bc-b329-6834fa2ccc3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.793292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f61b94-01c2-4a04-9865-13bf28280055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.793404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ea70e-be4d-46bc-b329-6834fa2ccc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.793860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ea70e-be4d-46bc-b329-6834fa2ccc3b admin admin] PATH_INFO: `/v3/projects/3c0cddb295c349f3b2a0ae4f513dbeca/users/de7fac78536d41c0a96b3c64e3a7b4fa/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.795143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f61b94-01c2-4a04-9865-13bf28280055 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.796512 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ea70e-be4d-46bc-b329-6834fa2ccc3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0cddb295c349f3b2a0ae4f513dbeca, user_id=de7fac78536d41c0a96b3c64e3a7b4fa, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.798996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ea70e-be4d-46bc-b329-6834fa2ccc3b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.809367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1305/2638] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:21.813233 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f61b94-01c2-4a04-9865-13bf28280055 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.815616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1334/2639] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:21 2022] PUT /identity/v3/projects/3c0cddb295c349f3b2a0ae4f513dbeca/users/de7fac78536d41c0a96b3c64e3a7b4fa/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:21.827874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b18a-e4c4-4085-ae8a-a4f39ecf3bc9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.828644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b18a-e4c4-4085-ae8a-a4f39ecf3bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.829047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29b18a-e4c4-4085-ae8a-a4f39ecf3bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.832654 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f61b94-01c2-4a04-9865-13bf28280055 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:21.863073 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-de29b18a-e4c4-4085-ae8a-a4f39ecf3bc9 None None] MFA Rules not processed for user `de7fac78536d41c0a96b3c64e3a7b4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:21.865294 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-de29b18a-e4c4-4085-ae8a-a4f39ecf3bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:21.885064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:21.886757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:21.908614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2f7cc-c864-4f30-add3-49527c69394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:21.909492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f7cc-c864-4f30-add3-49527c69394f admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.909778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f7cc-c864-4f30-add3-49527c69394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.910054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f7cc-c864-4f30-add3-49527c69394f admin admin] PATH_INFO: `/v3/projects/109e761904384877ae3802a2fbcfc4f4/users/401b110d3e3249dbbb4af206c9623eb1/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.925358 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2f7cc-c864-4f30-add3-49527c69394f admin admin] RBAC: Authorizing `identity:create_grant(project_id=109e761904384877ae3802a2fbcfc4f4, user_id=401b110d3e3249dbbb4af206c9623eb1, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:21.929118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2f7cc-c864-4f30-add3-49527c69394f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:21.943579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1306/2640] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:21 2022] PUT /identity/v3/projects/109e761904384877ae3802a2fbcfc4f4/users/401b110d3e3249dbbb4af206c9623eb1/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:21.946698 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1335/2641] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:21.954504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a587201-93c8-4f93-81e6-7a5215cd1c43 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:21.954798 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a587201-93c8-4f93-81e6-7a5215cd1c43 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:21.955072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a587201-93c8-4f93-81e6-7a5215cd1c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:21.958285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29b18a-e4c4-4085-ae8a-a4f39ecf3bc9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:21.959741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-de29b18a-e4c4-4085-ae8a-a4f39ecf3bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:21.982907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5a587201-93c8-4f93-81e6-7a5215cd1c43 None None] MFA Rules not processed for user `401b110d3e3249dbbb4af206c9623eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:21.984032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5a587201-93c8-4f93-81e6-7a5215cd1c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:22.023475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29b18a-e4c4-4085-ae8a-a4f39ecf3bc9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:22.062300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1307/2642] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:22.074425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a587201-93c8-4f93-81e6-7a5215cd1c43 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:22.075964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5a587201-93c8-4f93-81e6-7a5215cd1c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:22.081443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae447767-a2c1-41ee-81ec-627900931905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:22.082289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae447767-a2c1-41ee-81ec-627900931905 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:22.082441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae447767-a2c1-41ee-81ec-627900931905 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:22.082594 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae447767-a2c1-41ee-81ec-627900931905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:22.144868 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a587201-93c8-4f93-81e6-7a5215cd1c43 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:22.150347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39af41-bf39-4ca5-a824-7a09989aa8a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:22.150612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae447767-a2c1-41ee-81ec-627900931905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:22.151258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39af41-bf39-4ca5-a824-7a09989aa8a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:22.151720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39af41-bf39-4ca5-a824-7a09989aa8a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:22.152099 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39af41-bf39-4ca5-a824-7a09989aa8a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:22.152552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae447767-a2c1-41ee-81ec-627900931905 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:22.164179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39af41-bf39-4ca5-a824-7a09989aa8a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:22.165338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1336/2643] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:22.166259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d39af41-bf39-4ca5-a824-7a09989aa8a9 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:22.178202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1308/2644] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:22.218951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae447767-a2c1-41ee-81ec-627900931905 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:22.230890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae447767-a2c1-41ee-81ec-627900931905 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:22.235454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54087079-e6d6-4f5e-acaf-69bb948ecd7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:22.236168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54087079-e6d6-4f5e-acaf-69bb948ecd7e service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:22.236593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54087079-e6d6-4f5e-acaf-69bb948ecd7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:22.236957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54087079-e6d6-4f5e-acaf-69bb948ecd7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:22.239917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-54087079-e6d6-4f5e-acaf-69bb948ecd7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:22.264781 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:22.307981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54087079-e6d6-4f5e-acaf-69bb948ecd7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:22.309532 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54087079-e6d6-4f5e-acaf-69bb948ecd7e service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:22.320200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1337/2645] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:25.819835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39af41-bf39-4ca5-a824-7a09989aa8a9 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:25.878789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:25.885349 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39af41-bf39-4ca5-a824-7a09989aa8a9 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:25.890254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:25.890879 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:25.891158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:25.891432 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] PATH_INFO: `/v3/users/ef26a12440d840068314b36d5cdc1f21` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:25.892556 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef26a12440d840068314b36d5cdc1f21)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:25.894502 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:25.942757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef26a12440d840068314b36d5cdc1f21'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:25.950001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef26a12440d840068314b36d5cdc1f21'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:25.956803 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef26a12440d840068314b36d5cdc1f21'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:25.957436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef26a12440d840068314b36d5cdc1f21'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:25.958397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef26a12440d840068314b36d5cdc1f21'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:25.976237 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1309/2646] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:25 2022] DELETE /identity/v3/users/ef26a12440d840068314b36d5cdc1f21 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:25.980690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fea3b-6557-4503-a05e-eb131726ce63 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:25.981186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fea3b-6557-4503-a05e-eb131726ce63 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:25.981289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fea3b-6557-4503-a05e-eb131726ce63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:26.003054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fc2fea3b-6557-4503-a05e-eb131726ce63 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:26.004817 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fc2fea3b-6557-4503-a05e-eb131726ce63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:26.071938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1338/2647] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:26.081645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:26.083090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:26.138669 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f361f8d-adc2-449e-9296-91d91d067742 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:26.142407 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2f57-326c-4197-b45a-9d17fbf0ac53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:26.143051 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2f57-326c-4197-b45a-9d17fbf0ac53 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:26.143329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2f57-326c-4197-b45a-9d17fbf0ac53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:26.143600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2f57-326c-4197-b45a-9d17fbf0ac53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:26.153315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2f57-326c-4197-b45a-9d17fbf0ac53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:26.154738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2f57-326c-4197-b45a-9d17fbf0ac53 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:26.164789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1310/2648] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:26.437081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fea3b-6557-4503-a05e-eb131726ce63 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:26.438655 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fc2fea3b-6557-4503-a05e-eb131726ce63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:26.462124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fea3b-6557-4503-a05e-eb131726ce63 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:26.466814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:26.467447 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:26.467758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:26.468032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] PATH_INFO: `/v3/projects/b2bd6110dec1492e92cd550eb0071afb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:26.469332 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2bd6110dec1492e92cd550eb0071afb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:26.471250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:26.518268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2bd6110dec1492e92cd550eb0071afb', 'request_id': 'req-544cbf00-7d0e-49cb-b902-2384a3e5e14a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.524411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2bd6110dec1492e92cd550eb0071afb', 'request_id': 'req-544cbf00-7d0e-49cb-b902-2384a3e5e14a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.531945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2bd6110dec1492e92cd550eb0071afb', 'request_id': 'req-544cbf00-7d0e-49cb-b902-2384a3e5e14a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.543200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] The token cache is being invalidate because project b2bd6110dec1492e92cd550eb0071afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:26.543500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.544129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.545873 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1339/2649] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:26 2022] DELETE /identity/v3/projects/b2bd6110dec1492e92cd550eb0071afb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:26.549961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2f57-326c-4197-b45a-9d17fbf0ac53 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:26.559789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2f57-326c-4197-b45a-9d17fbf0ac53 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:26.615742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:26.616371 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:26.616679 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:26.616945 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] PATH_INFO: `/v3/users/de7fac78536d41c0a96b3c64e3a7b4fa` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:26.618061 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7fac78536d41c0a96b3c64e3a7b4fa)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:26.619959 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:26.640937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:26.666296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7fac78536d41c0a96b3c64e3a7b4fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.673727 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7fac78536d41c0a96b3c64e3a7b4fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.680956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7fac78536d41c0a96b3c64e3a7b4fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.681607 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7fac78536d41c0a96b3c64e3a7b4fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.682244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a11efac-9b37-449c-bfab-b3240bc6807f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7fac78536d41c0a96b3c64e3a7b4fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:26.700647 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1311/2650] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:26 2022] DELETE /identity/v3/users/de7fac78536d41c0a96b3c64e3a7b4fa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:26.973771 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:26.975543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:26.992940 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cbf00-7d0e-49cb-b902-2384a3e5e14a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.040170 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:27.058316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.058954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.059229 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.059500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] PATH_INFO: `/v3/projects/3c0cddb295c349f3b2a0ae4f513dbeca` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.060699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0cddb295c349f3b2a0ae4f513dbeca)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.062599 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.111372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0cddb295c349f3b2a0ae4f513dbeca', 'request_id': 'req-79e85b21-9a2e-4378-bee1-9e259d773fb6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:27.117371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0cddb295c349f3b2a0ae4f513dbeca', 'request_id': 'req-79e85b21-9a2e-4378-bee1-9e259d773fb6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:27.125025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0cddb295c349f3b2a0ae4f513dbeca', 'request_id': 'req-79e85b21-9a2e-4378-bee1-9e259d773fb6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:27.137046 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] The token cache is being invalidate because project 3c0cddb295c349f3b2a0ae4f513dbeca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:27.137337 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:27.138005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-79e85b21-9a2e-4378-bee1-9e259d773fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:27.139499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1340/2651] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:26 2022] DELETE /identity/v3/projects/3c0cddb295c349f3b2a0ae4f513dbeca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:27.144333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28401-f1f9-49e6-926d-e9eca215969a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.144809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28401-f1f9-49e6-926d-e9eca215969a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.144943 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28401-f1f9-49e6-926d-e9eca215969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.165784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-69d28401-f1f9-49e6-926d-e9eca215969a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:27.166975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-69d28401-f1f9-49e6-926d-e9eca215969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:27.228627 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1312/2652] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:27.236091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191fb16-a764-4be6-809b-18ed6de22250 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.236374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191fb16-a764-4be6-809b-18ed6de22250 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.236679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191fb16-a764-4be6-809b-18ed6de22250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.257852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d191fb16-a764-4be6-809b-18ed6de22250 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:27.259034 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d191fb16-a764-4be6-809b-18ed6de22250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:27.266775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1341/2653] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:27.271459 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d28401-f1f9-49e6-926d-e9eca215969a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.281224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d28401-f1f9-49e6-926d-e9eca215969a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.286513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.287210 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.287499 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.287779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.288519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:27.288823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.290405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.302865 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1313/2654] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:21:27.307261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191fb16-a764-4be6-809b-18ed6de22250 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.316558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191fb16-a764-4be6-809b-18ed6de22250 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.321616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ac9ca-0c0a-472d-af66-a239f4fda3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.322228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac9ca-0c0a-472d-af66-a239f4fda3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.322507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac9ca-0c0a-472d-af66-a239f4fda3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.322778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac9ca-0c0a-472d-af66-a239f4fda3c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.323403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ac9ca-0c0a-472d-af66-a239f4fda3c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.325384 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ac9ca-0c0a-472d-af66-a239f4fda3c2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.357111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1342/2655] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:27 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:27.361465 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.373128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3baf-97ae-4957-a536-25633e539f13 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.445537 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727f332-72b2-459d-a23a-f9597bdce8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.446292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727f332-72b2-459d-a23a-f9597bdce8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.446572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727f332-72b2-459d-a23a-f9597bdce8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.446848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727f332-72b2-459d-a23a-f9597bdce8c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.447485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727f332-72b2-459d-a23a-f9597bdce8c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.449829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727f332-72b2-459d-a23a-f9597bdce8c9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.472782 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1314/2656] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:27 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:27.477621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ac9ca-0c0a-472d-af66-a239f4fda3c2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.488173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ac9ca-0c0a-472d-af66-a239f4fda3c2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.493493 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cce86-eaa0-4876-b8cc-ce139a6e3d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.494083 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cce86-eaa0-4876-b8cc-ce139a6e3d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.494358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cce86-eaa0-4876-b8cc-ce139a6e3d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.494626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cce86-eaa0-4876-b8cc-ce139a6e3d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.495261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cce86-eaa0-4876-b8cc-ce139a6e3d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.496913 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cce86-eaa0-4876-b8cc-ce139a6e3d37 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.506828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1343/2657] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:27.510861 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727f332-72b2-459d-a23a-f9597bdce8c9 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.521146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727f332-72b2-459d-a23a-f9597bdce8c9 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.525902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a3585-c6c6-4bf0-bf71-3867a883fae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.526505 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a3585-c6c6-4bf0-bf71-3867a883fae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.526795 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a3585-c6c6-4bf0-bf71-3867a883fae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.527066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a3585-c6c6-4bf0-bf71-3867a883fae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.527710 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a3585-c6c6-4bf0-bf71-3867a883fae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.529328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a3585-c6c6-4bf0-bf71-3867a883fae4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.542328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1315/2658] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:27.545458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cce86-eaa0-4876-b8cc-ce139a6e3d37 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.556986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cce86-eaa0-4876-b8cc-ce139a6e3d37 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.562735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f826b25-0cbd-4c84-a20f-1afed0258953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.563410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826b25-0cbd-4c84-a20f-1afed0258953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.563706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826b25-0cbd-4c84-a20f-1afed0258953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.564122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826b25-0cbd-4c84-a20f-1afed0258953 admin admin] PATH_INFO: `/v3/projects/e1e70adda25d45dc9343787d3edcd093/users/da50cc2cf7554a1e91e837336ac26498/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.583652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f826b25-0cbd-4c84-a20f-1afed0258953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e70adda25d45dc9343787d3edcd093, user_id=da50cc2cf7554a1e91e837336ac26498, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.586376 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f826b25-0cbd-4c84-a20f-1afed0258953 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.599632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1344/2659] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:27 2022] PUT /identity/v3/projects/e1e70adda25d45dc9343787d3edcd093/users/da50cc2cf7554a1e91e837336ac26498/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:27.611263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc3469-529a-4ab9-994a-74f833d345b0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.611548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc3469-529a-4ab9-994a-74f833d345b0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.611821 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc3469-529a-4ab9-994a-74f833d345b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.622446 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:27.646807 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-51fc3469-529a-4ab9-994a-74f833d345b0 None None] MFA Rules not processed for user `da50cc2cf7554a1e91e837336ac26498`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:27.648011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-51fc3469-529a-4ab9-994a-74f833d345b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:27.717109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1316/2660] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:27.721803 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f826b25-0cbd-4c84-a20f-1afed0258953 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.803398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f826b25-0cbd-4c84-a20f-1afed0258953 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.809139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32294d42-f635-40e3-a984-373faf84fb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.809759 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294d42-f635-40e3-a984-373faf84fb28 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.810041 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294d42-f635-40e3-a984-373faf84fb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.810312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32294d42-f635-40e3-a984-373faf84fb28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.810943 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32294d42-f635-40e3-a984-373faf84fb28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.812917 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32294d42-f635-40e3-a984-373faf84fb28 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.829064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:27.847756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1345/2661] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:27 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:27.849649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc3469-529a-4ab9-994a-74f833d345b0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.860471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc3469-529a-4ab9-994a-74f833d345b0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.929668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f8e82-ea64-4ce6-a951-70aeee589117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.929668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f8e82-ea64-4ce6-a951-70aeee589117 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.929668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f8e82-ea64-4ce6-a951-70aeee589117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.929668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f8e82-ea64-4ce6-a951-70aeee589117 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.930184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f8e82-ea64-4ce6-a951-70aeee589117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.930184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f8e82-ea64-4ce6-a951-70aeee589117 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.954466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1317/2662] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:27 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:27.958669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32294d42-f635-40e3-a984-373faf84fb28 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:27.969422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32294d42-f635-40e3-a984-373faf84fb28 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:27.974523 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befb671-2bd2-4ca0-8814-f723c0339863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:27.975123 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befb671-2bd2-4ca0-8814-f723c0339863 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:27.975402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befb671-2bd2-4ca0-8814-f723c0339863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:27.975676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befb671-2bd2-4ca0-8814-f723c0339863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:27.976321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befb671-2bd2-4ca0-8814-f723c0339863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:27.977960 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befb671-2bd2-4ca0-8814-f723c0339863 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:27.993572 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1346/2663] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:27.994738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f8e82-ea64-4ce6-a951-70aeee589117 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:28.007848 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f8e82-ea64-4ce6-a951-70aeee589117 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:28.014201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712675a-c6bf-4860-925e-d75b87e77ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:28.014897 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712675a-c6bf-4860-925e-d75b87e77ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:28.015155 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712675a-c6bf-4860-925e-d75b87e77ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:28.015450 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712675a-c6bf-4860-925e-d75b87e77ab2 admin admin] PATH_INFO: `/v3/projects/3bfeebad00df4ab4ad2d491c3bb3d50a/users/d1c08f49e154433887903a4851c548c2/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:28.169220 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712675a-c6bf-4860-925e-d75b87e77ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bfeebad00df4ab4ad2d491c3bb3d50a, user_id=d1c08f49e154433887903a4851c548c2, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:28.172684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712675a-c6bf-4860-925e-d75b87e77ab2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:28.185809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1318/2664] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:27 2022] PUT /identity/v3/projects/3bfeebad00df4ab4ad2d491c3bb3d50a/users/d1c08f49e154433887903a4851c548c2/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:28.190131 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befb671-2bd2-4ca0-8814-f723c0339863 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:28.201053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befb671-2bd2-4ca0-8814-f723c0339863 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:28.264219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdf47b-e2b4-494a-a823-a4c5b035c794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:28.265103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf47b-e2b4-494a-a823-a4c5b035c794 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:28.265542 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf47b-e2b4-494a-a823-a4c5b035c794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:28.265810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf47b-e2b4-494a-a823-a4c5b035c794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:28.266443 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdf47b-e2b4-494a-a823-a4c5b035c794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:28.268011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdf47b-e2b4-494a-a823-a4c5b035c794 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:28.279139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1347/2665] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:28.283138 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712675a-c6bf-4860-925e-d75b87e77ab2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:28.294191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712675a-c6bf-4860-925e-d75b87e77ab2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:28.299109 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:28.299759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:28.300031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:28.300300 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] PATH_INFO: `/v3/projects/3bfeebad00df4ab4ad2d491c3bb3d50a/users/d1c08f49e154433887903a4851c548c2/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:28.302593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bfeebad00df4ab4ad2d491c3bb3d50a, user_id=d1c08f49e154433887903a4851c548c2, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:28.305258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:28.315437 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1319/2666] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:28 2022] PUT /identity/v3/projects/3bfeebad00df4ab4ad2d491c3bb3d50a/users/d1c08f49e154433887903a4851c548c2/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:28.326373 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5c18e-0c4a-4af4-be17-3fdf0a69d8cb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:28.326660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5c18e-0c4a-4af4-be17-3fdf0a69d8cb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:28.326936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5c18e-0c4a-4af4-be17-3fdf0a69d8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:28.359276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3ae5c18e-0c4a-4af4-be17-3fdf0a69d8cb None None] MFA Rules not processed for user `d1c08f49e154433887903a4851c548c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:28.360559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3ae5c18e-0c4a-4af4-be17-3fdf0a69d8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:28.438882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1348/2667] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:28.448801 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:28.450525 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:28.500058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:28.513179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca77aea-c9c3-47d6-bf1d-f7ee5d44faab admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:28.517393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6322e-a952-4031-9187-f77ce2e9dd82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:28.518114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6322e-a952-4031-9187-f77ce2e9dd82 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:28.518393 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6322e-a952-4031-9187-f77ce2e9dd82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:28.518660 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6322e-a952-4031-9187-f77ce2e9dd82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:28.581542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6322e-a952-4031-9187-f77ce2e9dd82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:28.583065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6322e-a952-4031-9187-f77ce2e9dd82 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:28.592314 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1320/2668] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:35.033460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5c18e-0c4a-4af4-be17-3fdf0a69d8cb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:35.065496 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5c18e-0c4a-4af4-be17-3fdf0a69d8cb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:35.085301 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69357470-c173-48fe-9067-209a952336fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:35.086128 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357470-c173-48fe-9067-209a952336fd service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:35.086526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357470-c173-48fe-9067-209a952336fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:35.086906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357470-c173-48fe-9067-209a952336fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:35.099113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69357470-c173-48fe-9067-209a952336fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:35.100610 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69357470-c173-48fe-9067-209a952336fd service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:35.111480 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1349/2669] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:38.588135 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=28 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:41.133077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961ded-2ab8-494a-831f-8de12b25a1d2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:41.133675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961ded-2ab8-494a-831f-8de12b25a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:41.133675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961ded-2ab8-494a-831f-8de12b25a1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:41.159908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ad961ded-2ab8-494a-831f-8de12b25a1d2 None None] MFA Rules not processed for user `4f7899ab15b347b7be75e3433f0d5450`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:41.161820 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ad961ded-2ab8-494a-831f-8de12b25a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:41.240490 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1321/2670] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:42.321815 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69357470-c173-48fe-9067-209a952336fd service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:42.358852 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad961ded-2ab8-494a-831f-8de12b25a1d2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:42.360728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ad961ded-2ab8-494a-831f-8de12b25a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:42.402054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:42.413199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69357470-c173-48fe-9067-209a952336fd service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:42.418889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:42.419870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:42.420144 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:42.420413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] PATH_INFO: `/v3/users/da50cc2cf7554a1e91e837336ac26498` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:42.421593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=da50cc2cf7554a1e91e837336ac26498)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:42.423562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:42.443524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad961ded-2ab8-494a-831f-8de12b25a1d2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:42.450055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:42.450806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:42.451310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:42.451404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] PATH_INFO: `/v3/users/401b110d3e3249dbbb4af206c9623eb1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:42.452677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=401b110d3e3249dbbb4af206c9623eb1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:42.455057 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:42.499140 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da50cc2cf7554a1e91e837336ac26498'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.515068 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da50cc2cf7554a1e91e837336ac26498'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.526894 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da50cc2cf7554a1e91e837336ac26498'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.527677 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da50cc2cf7554a1e91e837336ac26498'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.528473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76243db0-51dd-49c9-86c8-ecf0be35b4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da50cc2cf7554a1e91e837336ac26498'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.528990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '401b110d3e3249dbbb4af206c9623eb1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.540343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '401b110d3e3249dbbb4af206c9623eb1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.549271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '401b110d3e3249dbbb4af206c9623eb1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.553062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '401b110d3e3249dbbb4af206c9623eb1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.555010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1350/2671] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:42 2022] DELETE /identity/v3/users/da50cc2cf7554a1e91e837336ac26498 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:42.555676 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7c24ee79-1fbd-43dc-9a77-00cb4554e1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '401b110d3e3249dbbb4af206c9623eb1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:42.559624 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafb678-75c0-4955-ae6d-403a38244713 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:42.559904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafb678-75c0-4955-ae6d-403a38244713 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:42.560173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafb678-75c0-4955-ae6d-403a38244713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:42.579729 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1322/2672] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:42 2022] DELETE /identity/v3/users/401b110d3e3249dbbb4af206c9623eb1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:42.585267 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a265b4b-acc7-4f22-9db9-5491b7ee6fc0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:42.585768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a265b4b-acc7-4f22-9db9-5491b7ee6fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:42.586128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a265b4b-acc7-4f22-9db9-5491b7ee6fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:42.588567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-aaafb678-75c0-4955-ae6d-403a38244713 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:42.589794 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aaafb678-75c0-4955-ae6d-403a38244713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:42.610574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6a265b4b-acc7-4f22-9db9-5491b7ee6fc0 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:42.611800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a265b4b-acc7-4f22-9db9-5491b7ee6fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:42.636578 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:42.662377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1351/2673] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:42.674070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafb678-75c0-4955-ae6d-403a38244713 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:42.675520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aaafb678-75c0-4955-ae6d-403a38244713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:42.681830 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1323/2674] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:42.691221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a265b4b-acc7-4f22-9db9-5491b7ee6fc0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:42.693370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a265b4b-acc7-4f22-9db9-5491b7ee6fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:42.735342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafb678-75c0-4955-ae6d-403a38244713 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:42.740951 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:42.741575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:42.741900 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:42.742172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:42.755511 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:42.756827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:42.758601 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a265b4b-acc7-4f22-9db9-5491b7ee6fc0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:42.762980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:42.763699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:42.764039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:42.764378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:42.766693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1352/2675] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:42.777130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:42.778854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:42.789444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1324/2676] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:43.276836 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:43.278611 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:43.296626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30c449-4f0e-4575-83e0-6fda7bf41407 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:43.304739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:43.305691 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:43.306160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:43.306846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] PATH_INFO: `/v3/projects/109e761904384877ae3802a2fbcfc4f4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:43.308800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] RBAC: Authorizing `identity:delete_project(project_id=109e761904384877ae3802a2fbcfc4f4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:43.322547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:43.341912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:43.434436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:43.436683 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:43.448242 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '109e761904384877ae3802a2fbcfc4f4', 'request_id': 'req-60a1930c-7a74-4bec-ba8d-598cf578270c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.452055 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95802afb-28e4-4de3-be6b-6ec868defbf5 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:43.454536 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '109e761904384877ae3802a2fbcfc4f4', 'request_id': 'req-60a1930c-7a74-4bec-ba8d-598cf578270c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.464779 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '109e761904384877ae3802a2fbcfc4f4', 'request_id': 'req-60a1930c-7a74-4bec-ba8d-598cf578270c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.479297 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] The token cache is being invalidate because project 109e761904384877ae3802a2fbcfc4f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:43.479583 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.480263 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60a1930c-7a74-4bec-ba8d-598cf578270c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.481758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1353/2677] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:43 2022] DELETE /identity/v3/projects/109e761904384877ae3802a2fbcfc4f4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:43.487130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828deaa6-4b91-404c-b905-c65cb91eead4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:43.487414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828deaa6-4b91-404c-b905-c65cb91eead4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:43.487692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828deaa6-4b91-404c-b905-c65cb91eead4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:43.511192 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-828deaa6-4b91-404c-b905-c65cb91eead4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:43.512322 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-828deaa6-4b91-404c-b905-c65cb91eead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:43.525001 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:43.525825 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:43.526461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:43.526582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] PATH_INFO: `/v3/projects/e1e70adda25d45dc9343787d3edcd093` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:43.527770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e70adda25d45dc9343787d3edcd093)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:43.529930 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:43.584334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1354/2678] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:43.585401 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e70adda25d45dc9343787d3edcd093', 'request_id': 'req-9a2ed2a6-ea5a-431b-a75d-907f38b40544'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.592088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074cd20-ec33-4dd0-b083-568dad092ce4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:43.592570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074cd20-ec33-4dd0-b083-568dad092ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:43.592844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074cd20-ec33-4dd0-b083-568dad092ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:43.597657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e70adda25d45dc9343787d3edcd093', 'request_id': 'req-9a2ed2a6-ea5a-431b-a75d-907f38b40544'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.607364 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e70adda25d45dc9343787d3edcd093', 'request_id': 'req-9a2ed2a6-ea5a-431b-a75d-907f38b40544'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.616792 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:43.617021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4074cd20-ec33-4dd0-b083-568dad092ce4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:43.618735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4074cd20-ec33-4dd0-b083-568dad092ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:43.624680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] The token cache is being invalidate because project e1e70adda25d45dc9343787d3edcd093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:43.624984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.626251 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.628098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1325/2679] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:43 2022] DELETE /identity/v3/projects/e1e70adda25d45dc9343787d3edcd093 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:43.632940 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:43.634301 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:43.721146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1355/2680] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:43.721404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ed2a6-ea5a-431b-a75d-907f38b40544 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:43.722285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074cd20-ec33-4dd0-b083-568dad092ce4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:43.727058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:43.728042 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:43.728382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:43.728744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] PATH_INFO: `/v3/users/d1c08f49e154433887903a4851c548c2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:43.729941 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c08f49e154433887903a4851c548c2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:43.731839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074cd20-ec33-4dd0-b083-568dad092ce4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:43.731947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:43.736627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:43.737223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:43.737505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:43.737783 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:43.739096 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:43.739662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:43.741478 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:43.755890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1356/2681] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:21:43.760703 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:43.772668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd291-c229-4fba-ad3b-439220b35242 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:43.777528 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:43.778137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:43.778407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:43.778690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:43.779300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:43.781265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:43.792809 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c08f49e154433887903a4851c548c2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.804370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c08f49e154433887903a4851c548c2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.813461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c08f49e154433887903a4851c548c2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.814166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c08f49e154433887903a4851c548c2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.814969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c08f49e154433887903a4851c548c2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:43.823713 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1357/2682] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:43 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:43.825945 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:43.829111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:43.839018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1326/2683] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:43 2022] DELETE /identity/v3/users/d1c08f49e154433887903a4851c548c2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:43.842111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:43.946563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc27adb-421b-4da0-87e9-0f1794b09c35 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.016497 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7aaaa-7dd2-4b01-b307-e631b9b576b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.017241 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7aaaa-7dd2-4b01-b307-e631b9b576b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.017526 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7aaaa-7dd2-4b01-b307-e631b9b576b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.017796 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7aaaa-7dd2-4b01-b307-e631b9b576b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.018555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7aaaa-7dd2-4b01-b307-e631b9b576b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.021283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7aaaa-7dd2-4b01-b307-e631b9b576b5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.046309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1358/2684] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:43 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:44.050195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.061596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e904-3b08-4f98-8659-de3fff88556a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.067214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f62d3a-8d07-442c-91df-2a1d0efb3240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.068034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62d3a-8d07-442c-91df-2a1d0efb3240 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.068557 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62d3a-8d07-442c-91df-2a1d0efb3240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.068695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62d3a-8d07-442c-91df-2a1d0efb3240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.069404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f62d3a-8d07-442c-91df-2a1d0efb3240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.071231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f62d3a-8d07-442c-91df-2a1d0efb3240 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.084902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1327/2685] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:21:44.086509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7aaaa-7dd2-4b01-b307-e631b9b576b5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.097199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7aaaa-7dd2-4b01-b307-e631b9b576b5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.105368 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.105985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.106260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.106535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.109425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.109425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.120078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1359/2686] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:44.124628 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f62d3a-8d07-442c-91df-2a1d0efb3240 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.140002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f62d3a-8d07-442c-91df-2a1d0efb3240 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.146729 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd8a12-488e-4bfd-bbf8-7b973c6c583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.147427 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8a12-488e-4bfd-bbf8-7b973c6c583f admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.148372 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8a12-488e-4bfd-bbf8-7b973c6c583f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.148702 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8a12-488e-4bfd-bbf8-7b973c6c583f admin admin] PATH_INFO: `/v3/projects/046e8c5311ab49d08650d9e1eeadf761/users/b3e1a8c84c074c8791b7ef6316ef1542/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.161816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.164952 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.169887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd8a12-488e-4bfd-bbf8-7b973c6c583f admin admin] RBAC: Authorizing `identity:create_grant(project_id=046e8c5311ab49d08650d9e1eeadf761, user_id=b3e1a8c84c074c8791b7ef6316ef1542, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.177031 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd8a12-488e-4bfd-bbf8-7b973c6c583f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.198524 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02bdff-dff5-42df-847a-0462cae999a1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.204507 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1328/2687] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:44 2022] PUT /identity/v3/projects/046e8c5311ab49d08650d9e1eeadf761/users/b3e1a8c84c074c8791b7ef6316ef1542/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:44.213778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b507e-321e-428a-864b-85be55bcefc6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.213778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b507e-321e-428a-864b-85be55bcefc6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.214102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b507e-321e-428a-864b-85be55bcefc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.262498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c68b507e-321e-428a-864b-85be55bcefc6 None None] MFA Rules not processed for user `b3e1a8c84c074c8791b7ef6316ef1542`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:44.264181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c68b507e-321e-428a-864b-85be55bcefc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.290537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.291185 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.291464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.291741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] PATH_INFO: `/v3/projects/3bfeebad00df4ab4ad2d491c3bb3d50a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.292990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bfeebad00df4ab4ad2d491c3bb3d50a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.295321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.304209 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:44.344848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:44.361668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bfeebad00df4ab4ad2d491c3bb3d50a', 'request_id': 'req-3cc8527a-9205-402b-a9ef-33e6c8443dfd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.362338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1329/2688] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:44.370789 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bfeebad00df4ab4ad2d491c3bb3d50a', 'request_id': 'req-3cc8527a-9205-402b-a9ef-33e6c8443dfd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.375821 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b507e-321e-428a-864b-85be55bcefc6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.377367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c68b507e-321e-428a-864b-85be55bcefc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.394608 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bfeebad00df4ab4ad2d491c3bb3d50a', 'request_id': 'req-3cc8527a-9205-402b-a9ef-33e6c8443dfd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.420194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] The token cache is being invalidate because project 3bfeebad00df4ab4ad2d491c3bb3d50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:44.420592 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.421220 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cc8527a-9205-402b-a9ef-33e6c8443dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.423095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1360/2689] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:44 2022] DELETE /identity/v3/projects/3bfeebad00df4ab4ad2d491c3bb3d50a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:44.429494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927b4c5-0ac8-4e9a-b319-e34ab6dfc609 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.429777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927b4c5-0ac8-4e9a-b319-e34ab6dfc609 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.430052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927b4c5-0ac8-4e9a-b319-e34ab6dfc609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.455751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7927b4c5-0ac8-4e9a-b319-e34ab6dfc609 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:44.457622 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7927b4c5-0ac8-4e9a-b319-e34ab6dfc609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.486508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b507e-321e-428a-864b-85be55bcefc6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.546604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1361/2690] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:44.551925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.552736 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.553015 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.553292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.555344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.556375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae85c3-d0ec-43cd-a3ff-7e3da9afebb4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.556697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae85c3-d0ec-43cd-a3ff-7e3da9afebb4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.557226 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae85c3-d0ec-43cd-a3ff-7e3da9afebb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.582132 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1dae85c3-d0ec-43cd-a3ff-7e3da9afebb4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:44.584595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1dae85c3-d0ec-43cd-a3ff-7e3da9afebb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.605365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1362/2691] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:44.607014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae85c3-d0ec-43cd-a3ff-7e3da9afebb4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.620171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae85c3-d0ec-43cd-a3ff-7e3da9afebb4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.626644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.627863 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.627995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.628664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.630534 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:44.630853 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.633156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.640346 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.642271 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.647133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1363/2692] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:21:44.651182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.654503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1330/2693] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:21:44.661551 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda5a9-fd85-4434-bdc9-437495079027 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.666353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4713b-b9ca-4e8b-90ed-9f6028fe637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.666964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4713b-b9ca-4e8b-90ed-9f6028fe637a admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.667307 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4713b-b9ca-4e8b-90ed-9f6028fe637a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.667642 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4713b-b9ca-4e8b-90ed-9f6028fe637a admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.668334 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4713b-b9ca-4e8b-90ed-9f6028fe637a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.671052 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4713b-b9ca-4e8b-90ed-9f6028fe637a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.685030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.686227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.701650 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827221af-4657-48d4-9aab-af61c27d2885 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.706972 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.710664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1364/2694] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:44 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:44.710846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.710846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.710846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] PATH_INFO: `/v3/users/b3e1a8c84c074c8791b7ef6316ef1542` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.710846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e1a8c84c074c8791b7ef6316ef1542)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.711871 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.712013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4713b-b9ca-4e8b-90ed-9f6028fe637a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.761027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:44.772799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e1a8c84c074c8791b7ef6316ef1542'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.781250 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e1a8c84c074c8791b7ef6316ef1542'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.789651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e1a8c84c074c8791b7ef6316ef1542'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.790227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4713b-b9ca-4e8b-90ed-9f6028fe637a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.790356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e1a8c84c074c8791b7ef6316ef1542'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.791094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e128318-ad30-4bae-a105-d6026384eaaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e1a8c84c074c8791b7ef6316ef1542'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:44.795398 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.796002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.796282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.796586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.797223 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.799152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.820259 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1331/2695] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:21:44 2022] DELETE /identity/v3/users/b3e1a8c84c074c8791b7ef6316ef1542 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:21:44.824884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a2d81-5fd6-44ce-9913-f1f84d9a86e0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.825582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a2d81-5fd6-44ce-9913-f1f84d9a86e0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.825921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a2d81-5fd6-44ce-9913-f1f84d9a86e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.828545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1365/2696] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:44 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:44.833672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.835372 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.847589 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8366a-8040-4513-aeb9-8b5eda124281 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.850077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-fa7a2d81-5fd6-44ce-9913-f1f84d9a86e0 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:44.852133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fa7a2d81-5fd6-44ce-9913-f1f84d9a86e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.853925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:44.923234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba6603-9712-4479-b12d-5c335ea673d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.923837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba6603-9712-4479-b12d-5c335ea673d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.924110 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba6603-9712-4479-b12d-5c335ea673d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.924379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba6603-9712-4479-b12d-5c335ea673d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.925067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba6603-9712-4479-b12d-5c335ea673d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.926935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1332/2697] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:21:44.929175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba6603-9712-4479-b12d-5c335ea673d3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:44.938442 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a2d81-5fd6-44ce-9913-f1f84d9a86e0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.939949 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fa7a2d81-5fd6-44ce-9913-f1f84d9a86e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:44.948219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1366/2698] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:21:44.953449 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba6603-9712-4479-b12d-5c335ea673d3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:44.965587 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba6603-9712-4479-b12d-5c335ea673d3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:44.972156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd2c60-8031-44be-a86d-87bfab8abc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:44.973545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd2c60-8031-44be-a86d-87bfab8abc17 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:44.974194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd2c60-8031-44be-a86d-87bfab8abc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:44.974720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd2c60-8031-44be-a86d-87bfab8abc17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:44.975748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd2c60-8031-44be-a86d-87bfab8abc17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:44.993972 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd2c60-8031-44be-a86d-87bfab8abc17 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:45.011295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1367/2699] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:45.012656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd2c60-8031-44be-a86d-87bfab8abc17 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:45.018064 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a2d81-5fd6-44ce-9913-f1f84d9a86e0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:45.024709 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd2c60-8031-44be-a86d-87bfab8abc17 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:45.024885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ca341-3d58-43cc-a871-fc8cdffcf0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:45.025404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ca341-3d58-43cc-a871-fc8cdffcf0be service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.025856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ca341-3d58-43cc-a871-fc8cdffcf0be service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.026129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ca341-3d58-43cc-a871-fc8cdffcf0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.033821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:45.034773 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.036419 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.037466 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] PATH_INFO: `/v3/projects/7a98deae424a4aa990a7c78111cbea70/users/66df6c7610de4cf390419278c528ab77/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.045077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ca341-3d58-43cc-a871-fc8cdffcf0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:45.046671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ca341-3d58-43cc-a871-fc8cdffcf0be service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:45.061133 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a98deae424a4aa990a7c78111cbea70, user_id=66df6c7610de4cf390419278c528ab77, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:45.065311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:45.067125 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1333/2700] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:45.084530 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1368/2701] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:45 2022] PUT /identity/v3/projects/7a98deae424a4aa990a7c78111cbea70/users/66df6c7610de4cf390419278c528ab77/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:45.098615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e76d02-efe3-4913-9e25-4f25f80aa762 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.099292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e76d02-efe3-4913-9e25-4f25f80aa762 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.099609 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e76d02-efe3-4913-9e25-4f25f80aa762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.134120 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-13e76d02-efe3-4913-9e25-4f25f80aa762 None None] MFA Rules not processed for user `66df6c7610de4cf390419278c528ab77`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:45.135692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-13e76d02-efe3-4913-9e25-4f25f80aa762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:45.194662 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:45.217366 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1334/2702] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:45.230143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:45.232374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:45.314629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86f946-cb3c-4829-bd43-af4c459a692d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:45.318675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee5719-ba0e-4879-affa-61c7adc2170b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a88817f3a4402290e9358826a63904', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:45.319277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee5719-ba0e-4879-affa-61c7adc2170b service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.319560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee5719-ba0e-4879-affa-61c7adc2170b service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.319828 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee5719-ba0e-4879-affa-61c7adc2170b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.329064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee5719-ba0e-4879-affa-61c7adc2170b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:45.330546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee5719-ba0e-4879-affa-61c7adc2170b service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:45.340887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1369/2703] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:45.565985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e76d02-efe3-4913-9e25-4f25f80aa762 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:45.568744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-13e76d02-efe3-4913-9e25-4f25f80aa762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:45.662095 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e76d02-efe3-4913-9e25-4f25f80aa762 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:45.667435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:45.668630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.669044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.669718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] PATH_INFO: `/v3/projects/046e8c5311ab49d08650d9e1eeadf761` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.671128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=046e8c5311ab49d08650d9e1eeadf761)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:45.673472 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:45.736466 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046e8c5311ab49d08650d9e1eeadf761', 'request_id': 'req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:45.745026 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046e8c5311ab49d08650d9e1eeadf761', 'request_id': 'req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:45.757506 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046e8c5311ab49d08650d9e1eeadf761', 'request_id': 'req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:45.775995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] The token cache is being invalidate because project 046e8c5311ab49d08650d9e1eeadf761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:21:45.776285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:45.776956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ff84c01e-bc90-47d3-bd9a-9ccf561dffbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:21:45.778768 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1335/2704] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:21:45 2022] DELETE /identity/v3/projects/046e8c5311ab49d08650d9e1eeadf761 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:45.784017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadacd3-9e87-40bf-a64b-af63e8f2320a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.784574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadacd3-9e87-40bf-a64b-af63e8f2320a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.784574 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadacd3-9e87-40bf-a64b-af63e8f2320a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.807196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0cadacd3-9e87-40bf-a64b-af63e8f2320a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:45.808421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0cadacd3-9e87-40bf-a64b-af63e8f2320a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:45.882356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1370/2705] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:45.890480 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e8c52-f961-496f-899c-f38dad3783b4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.890763 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e8c52-f961-496f-899c-f38dad3783b4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.891196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e8c52-f961-496f-899c-f38dad3783b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.918273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-688e8c52-f961-496f-899c-f38dad3783b4 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:45.919770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-688e8c52-f961-496f-899c-f38dad3783b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:45.929471 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1336/2706] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:45.934057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cadacd3-9e87-40bf-a64b-af63e8f2320a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:45.946616 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cadacd3-9e87-40bf-a64b-af63e8f2320a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:45.952432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:45.953379 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.953908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.954151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.954872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:21:45.955156 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:45.957011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:45.967152 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:45.970822 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1371/2707] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:21:45.975305 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e8c52-f961-496f-899c-f38dad3783b4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:45.984829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e8c52-f961-496f-899c-f38dad3783b4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:45.991543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc53dd3-81b3-4464-9632-e142bce9fe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:45.992264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc53dd3-81b3-4464-9632-e142bce9fe97 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:45.992573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc53dd3-81b3-4464-9632-e142bce9fe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:45.992860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc53dd3-81b3-4464-9632-e142bce9fe97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:45.993503 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc53dd3-81b3-4464-9632-e142bce9fe97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:45.995430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc53dd3-81b3-4464-9632-e142bce9fe97 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:46.024482 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1337/2708] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:21:45 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:21:46.028776 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:46.039711 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d73ac-31b7-4719-8f94-5d0472699f7b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:46.101409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb1e5-2da4-487a-a360-55bade2fff25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:46.102017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb1e5-2da4-487a-a360-55bade2fff25 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:46.102296 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb1e5-2da4-487a-a360-55bade2fff25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:46.102567 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbb1e5-2da4-487a-a360-55bade2fff25 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:46.103411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbb1e5-2da4-487a-a360-55bade2fff25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:46.105473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbb1e5-2da4-487a-a360-55bade2fff25 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:46.138350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1372/2709] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:21:46 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:21:46.139171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc53dd3-81b3-4464-9632-e142bce9fe97 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:46.150759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc53dd3-81b3-4464-9632-e142bce9fe97 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:46.155976 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468ae60-5d1c-46a6-9b16-bef08526c02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:46.156728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468ae60-5d1c-46a6-9b16-bef08526c02b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:46.157153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468ae60-5d1c-46a6-9b16-bef08526c02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:46.157435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468ae60-5d1c-46a6-9b16-bef08526c02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:46.158189 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468ae60-5d1c-46a6-9b16-bef08526c02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:46.160991 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468ae60-5d1c-46a6-9b16-bef08526c02b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:46.176391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1338/2710] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:46.178504 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb1e5-2da4-487a-a360-55bade2fff25 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:46.190827 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbb1e5-2da4-487a-a360-55bade2fff25 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:46.196604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873537f-7b40-46ca-b6ff-1520e1374f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:46.197154 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873537f-7b40-46ca-b6ff-1520e1374f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:46.197502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873537f-7b40-46ca-b6ff-1520e1374f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:46.197850 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873537f-7b40-46ca-b6ff-1520e1374f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:46.198705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873537f-7b40-46ca-b6ff-1520e1374f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:46.200649 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873537f-7b40-46ca-b6ff-1520e1374f8a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:46.219514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1373/2711] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:21:46.224950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468ae60-5d1c-46a6-9b16-bef08526c02b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:46.237166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468ae60-5d1c-46a6-9b16-bef08526c02b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:46.243288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:46.243957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:46.244254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:46.244549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] PATH_INFO: `/v3/projects/fb6091337b5842b4bcbad8e1b30f7b57/users/8ebd5a14c4d54921ad1b265be545adbd/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:46.270291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6091337b5842b4bcbad8e1b30f7b57, user_id=8ebd5a14c4d54921ad1b265be545adbd, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:46.273642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:46.291263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1339/2712] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:21:46 2022] PUT /identity/v3/projects/fb6091337b5842b4bcbad8e1b30f7b57/users/8ebd5a14c4d54921ad1b265be545adbd/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:21:46.302577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ee3ca-6ae0-47bf-876a-6be6c95cdedd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:46.304614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ee3ca-6ae0-47bf-876a-6be6c95cdedd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:46.305250 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ee3ca-6ae0-47bf-876a-6be6c95cdedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:46.340538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c25ee3ca-6ae0-47bf-876a-6be6c95cdedd None None] MFA Rules not processed for user `8ebd5a14c4d54921ad1b265be545adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:46.341777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c25ee3ca-6ae0-47bf-876a-6be6c95cdedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:46.422859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1374/2713] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:46.434166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:46.435863 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:46.450255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:46.498664 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f7b93-9552-4efb-9311-71c943507029 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:46.502326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:46.502931 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:46.503243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:46.503522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:46.512584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:46.514007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:46.523213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1340/2714] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:47.958919 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f4bfa-da1f-45a4-a3d2-2ad40926ded0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:47.959708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f4bfa-da1f-45a4-a3d2-2ad40926ded0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:47.959708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f4bfa-da1f-45a4-a3d2-2ad40926ded0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:47.991495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3a1f4bfa-da1f-45a4-a3d2-2ad40926ded0 None None] MFA Rules not processed for user `8ebd5a14c4d54921ad1b265be545adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:21:47.992737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3a1f4bfa-da1f-45a4-a3d2-2ad40926ded0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:48.019475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1375/2715] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:21:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:21:53.897257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:53.898988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:53.989610 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9afd7-ac56-4ed4-ac31-0d4fe0e7b80e service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:53.993901 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d73b90-464b-4981-93ad-e69256543d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:53.994504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73b90-464b-4981-93ad-e69256543d8e service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:53.994778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73b90-464b-4981-93ad-e69256543d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:53.995050 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73b90-464b-4981-93ad-e69256543d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:53.996885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-22d73b90-464b-4981-93ad-e69256543d8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:54.071054 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d73b90-464b-4981-93ad-e69256543d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:54.072916 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d73b90-464b-4981-93ad-e69256543d8e service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:54.084697 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1341/2716] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:21:56.887288 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f4bfa-da1f-45a4-a3d2-2ad40926ded0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:21:56.889899 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3a1f4bfa-da1f-45a4-a3d2-2ad40926ded0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:56.939071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:21:57.002571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f4bfa-da1f-45a4-a3d2-2ad40926ded0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:21:57.014418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:21:57.016331 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:21:57.016821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:21:57.017290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:21:57.020257 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:21:57.040313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:21:57.044103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:21:57.068904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1376/2717] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:21:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:22:03.807522 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606574-2ea6-4b37-99f2-28d32cc25eb9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:03.808040 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606574-2ea6-4b37-99f2-28d32cc25eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:03.808144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606574-2ea6-4b37-99f2-28d32cc25eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:03.837494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b1606574-2ea6-4b37-99f2-28d32cc25eb9 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:03.839201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b1606574-2ea6-4b37-99f2-28d32cc25eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:03.867467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:03.904170 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1342/2718] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:22:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:03.950500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:03.951933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:04.021591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3368d-2148-4724-b887-3feb5d26b130 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:04.026603 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7d286-49a1-4897-8603-585f46cf11cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:04.027199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7d286-49a1-4897-8603-585f46cf11cc service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:04.027479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7d286-49a1-4897-8603-585f46cf11cc service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:04.027750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7d286-49a1-4897-8603-585f46cf11cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:04.035562 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7d286-49a1-4897-8603-585f46cf11cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:04.038182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7d286-49a1-4897-8603-585f46cf11cc service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:04.046303 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1377/2719] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:22:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:22:04.568810 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31bc5-d033-42e9-af6a-e5c7aede7e60 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:04.569094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31bc5-d033-42e9-af6a-e5c7aede7e60 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:04.569367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31bc5-d033-42e9-af6a-e5c7aede7e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:04.588356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a8b31bc5-d033-42e9-af6a-e5c7aede7e60 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:04.589580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a8b31bc5-d033-42e9-af6a-e5c7aede7e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:04.597152 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1343/2720] 10.222.0.70 () {58 vars in 967 bytes} [Tue Aug 9 19:22:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:04.610440 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7d286-49a1-4897-8603-585f46cf11cc service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:04.622125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7d286-49a1-4897-8603-585f46cf11cc service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:04.627120 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d170fc-85a2-498d-ae79-78d4d8bf732b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:04.627705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d170fc-85a2-498d-ae79-78d4d8bf732b service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:04.627979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d170fc-85a2-498d-ae79-78d4d8bf732b service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:04.628271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d170fc-85a2-498d-ae79-78d4d8bf732b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:04.637167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d170fc-85a2-498d-ae79-78d4d8bf732b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:04.638719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d170fc-85a2-498d-ae79-78d4d8bf732b service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:04.647100 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1378/2721] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:22:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:22:04.866543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d3f44-bd77-4094-b948-48118070feff None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:04.867017 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d3f44-bd77-4094-b948-48118070feff None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:04.867122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d3f44-bd77-4094-b948-48118070feff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:04.886091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-970d3f44-bd77-4094-b948-48118070feff None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:04.887266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-970d3f44-bd77-4094-b948-48118070feff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:04.894762 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1344/2722] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Aug 9 19:22:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:04.906769 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d170fc-85a2-498d-ae79-78d4d8bf732b service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:04.918402 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d170fc-85a2-498d-ae79-78d4d8bf732b service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:04.923585 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34aa8-19d2-4a3b-80d3-3f953d27c90d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:04.924169 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34aa8-19d2-4a3b-80d3-3f953d27c90d service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:04.924470 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34aa8-19d2-4a3b-80d3-3f953d27c90d service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:04.924751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34aa8-19d2-4a3b-80d3-3f953d27c90d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:04.933606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f34aa8-19d2-4a3b-80d3-3f953d27c90d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:04.935168 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f34aa8-19d2-4a3b-80d3-3f953d27c90d service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:04.943413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1379/2723] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:22:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:22:05.268348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d653b1-4a32-450f-bf53-9add08c6e057 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:05.268658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d653b1-4a32-450f-bf53-9add08c6e057 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:05.268934 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d653b1-4a32-450f-bf53-9add08c6e057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:05.288380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a9d653b1-4a32-450f-bf53-9add08c6e057 None None] MFA Rules not processed for user `87b647f3da7246b0b530a65c500d5064`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:05.289580 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a9d653b1-4a32-450f-bf53-9add08c6e057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:05.296882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1345/2724] 10.222.0.70 () {58 vars in 967 bytes} [Tue Aug 9 19:22:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:05.308898 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34aa8-19d2-4a3b-80d3-3f953d27c90d service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:05.319731 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34aa8-19d2-4a3b-80d3-3f953d27c90d service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:05.325193 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f214d0-9a4b-45c7-8ee6-428f26e47fe3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '141d2c2f92c842feb6155aae1e9edfd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:05.325413 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f214d0-9a4b-45c7-8ee6-428f26e47fe3 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:05.325693 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f214d0-9a4b-45c7-8ee6-428f26e47fe3 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:05.325970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f214d0-9a4b-45c7-8ee6-428f26e47fe3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:05.334760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f214d0-9a4b-45c7-8ee6-428f26e47fe3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:05.336323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f214d0-9a4b-45c7-8ee6-428f26e47fe3 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:05.344186 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1380/2725] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Aug 9 19:22:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 19:22:07.434326 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d653b1-4a32-450f-bf53-9add08c6e057 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:07.517911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d653b1-4a32-450f-bf53-9add08c6e057 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:07.522919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:07.523059 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:07.523333 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:07.523624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] PATH_INFO: `/v3/users/8ebd5a14c4d54921ad1b265be545adbd` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:07.524749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebd5a14c4d54921ad1b265be545adbd)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:07.526626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:07.571623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebd5a14c4d54921ad1b265be545adbd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:07.578605 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebd5a14c4d54921ad1b265be545adbd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:07.585288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebd5a14c4d54921ad1b265be545adbd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:07.585927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebd5a14c4d54921ad1b265be545adbd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:07.586520 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebd5a14c4d54921ad1b265be545adbd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:07.604287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1346/2726] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:22:07 2022] DELETE /identity/v3/users/8ebd5a14c4d54921ad1b265be545adbd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:22:07.608823 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5c0ac-7058-4915-bfeb-a485d56f0549 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:07.609299 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5c0ac-7058-4915-bfeb-a485d56f0549 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:07.609400 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5c0ac-7058-4915-bfeb-a485d56f0549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:07.649498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f8c5c0ac-7058-4915-bfeb-a485d56f0549 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:07.650707 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f8c5c0ac-7058-4915-bfeb-a485d56f0549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:07.682600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:07.710874 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1381/2727] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:07.720859 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:07.722260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:07.735646 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:07.802086 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a423389-a7ca-4754-85c7-ef9591a9d622 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:07.805706 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:07.806299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:07.806570 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:07.806836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:07.815453 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:07.816896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:07.826630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1347/2728] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:22:08.000689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5c0ac-7058-4915-bfeb-a485d56f0549 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:08.003508 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f8c5c0ac-7058-4915-bfeb-a485d56f0549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:08.018910 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5c0ac-7058-4915-bfeb-a485d56f0549 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:08.030293 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:08.032130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:08.032343 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:08.032647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] PATH_INFO: `/v3/projects/fb6091337b5842b4bcbad8e1b30f7b57` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:08.034059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6091337b5842b4bcbad8e1b30f7b57)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:08.037065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:08.084727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6091337b5842b4bcbad8e1b30f7b57', 'request_id': 'req-3cf66dd7-a528-4735-a768-3c5d26a3dd49'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:08.090258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6091337b5842b4bcbad8e1b30f7b57', 'request_id': 'req-3cf66dd7-a528-4735-a768-3c5d26a3dd49'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:08.098632 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6091337b5842b4bcbad8e1b30f7b57', 'request_id': 'req-3cf66dd7-a528-4735-a768-3c5d26a3dd49'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:08.110495 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] The token cache is being invalidate because project fb6091337b5842b4bcbad8e1b30f7b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:22:08.110782 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:08.111450 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cf66dd7-a528-4735-a768-3c5d26a3dd49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:08.113005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1382/2729] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:22:07 2022] DELETE /identity/v3/projects/fb6091337b5842b4bcbad8e1b30f7b57 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:22:11.331020 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:11.332644 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:11.346082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04894bfb-aee0-47cc-8f6e-1110a65ddc21 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:11.403766 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:11.404484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:11.404771 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:11.405046 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] PATH_INFO: `/v3/users/66df6c7610de4cf390419278c528ab77` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:11.406176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66df6c7610de4cf390419278c528ab77)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:11.408079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:11.454749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66df6c7610de4cf390419278c528ab77'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:11.462552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66df6c7610de4cf390419278c528ab77'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:11.469428 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66df6c7610de4cf390419278c528ab77'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:11.470075 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66df6c7610de4cf390419278c528ab77'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:11.470684 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66df6c7610de4cf390419278c528ab77'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:11.488365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1348/2730] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:22:11 2022] DELETE /identity/v3/users/66df6c7610de4cf390419278c528ab77 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:22:11.493021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136aa7-c53a-4cd5-b7f0-8f4d4e522868 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:11.493498 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136aa7-c53a-4cd5-b7f0-8f4d4e522868 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:11.493595 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136aa7-c53a-4cd5-b7f0-8f4d4e522868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:11.512963 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-79136aa7-c53a-4cd5-b7f0-8f4d4e522868 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:11.514177 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-79136aa7-c53a-4cd5-b7f0-8f4d4e522868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:11.572381 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1383/2731] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:11.582370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:11.584375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:11.599126 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:11.643565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f1c11-717c-4781-a165-a210d77edcf2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:11.647937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef423e3d-dbaa-455d-81c1-08b97357d45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:11.648568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef423e3d-dbaa-455d-81c1-08b97357d45b service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:11.648849 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef423e3d-dbaa-455d-81c1-08b97357d45b service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:11.649129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef423e3d-dbaa-455d-81c1-08b97357d45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:11.658006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef423e3d-dbaa-455d-81c1-08b97357d45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:11.659413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef423e3d-dbaa-455d-81c1-08b97357d45b service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:11.668938 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1349/2732] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:22:11.941295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79136aa7-c53a-4cd5-b7f0-8f4d4e522868 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:11.943277 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-79136aa7-c53a-4cd5-b7f0-8f4d4e522868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:11.955593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79136aa7-c53a-4cd5-b7f0-8f4d4e522868 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:11.962954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:11.963821 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:11.964105 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:11.964377 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] PATH_INFO: `/v3/projects/7a98deae424a4aa990a7c78111cbea70` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:11.965557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a98deae424a4aa990a7c78111cbea70)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:11.967464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:12.015563 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a98deae424a4aa990a7c78111cbea70', 'request_id': 'req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:12.019986 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:12.021795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a98deae424a4aa990a7c78111cbea70', 'request_id': 'req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:12.029558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a98deae424a4aa990a7c78111cbea70', 'request_id': 'req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:12.042181 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] The token cache is being invalidate because project 7a98deae424a4aa990a7c78111cbea70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:22:12.042460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:12.043078 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-56e496d5-6426-4a6b-89ea-0916a3f9a8bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:12.044835 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1384/2733] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:22:11 2022] DELETE /identity/v3/projects/7a98deae424a4aa990a7c78111cbea70 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:22:12.049678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea052f-b853-4d43-b8ee-cee777f44d0e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.049947 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea052f-b853-4d43-b8ee-cee777f44d0e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.050224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea052f-b853-4d43-b8ee-cee777f44d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.097728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-40ea052f-b853-4d43-b8ee-cee777f44d0e None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:12.099645 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-40ea052f-b853-4d43-b8ee-cee777f44d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:12.159805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1350/2734] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:12.168538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa9c96-e010-43ca-86ec-9b7c6379d8c7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.168824 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa9c96-e010-43ca-86ec-9b7c6379d8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.169356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa9c96-e010-43ca-86ec-9b7c6379d8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.189090 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b6fa9c96-e010-43ca-86ec-9b7c6379d8c7 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:12.190291 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b6fa9c96-e010-43ca-86ec-9b7c6379d8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:12.198885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1385/2735] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:12.203087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea052f-b853-4d43-b8ee-cee777f44d0e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:12.212794 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea052f-b853-4d43-b8ee-cee777f44d0e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:12.217433 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:12.218021 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.218292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.218560 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.219195 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:22:12.219494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:12.221103 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:12.232649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1351/2736] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:22:12.237089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa9c96-e010-43ca-86ec-9b7c6379d8c7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:12.247072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa9c96-e010-43ca-86ec-9b7c6379d8c7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:12.251674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49c285-13d4-4c21-9b2e-289187a1017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:12.252261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c285-13d4-4c21-9b2e-289187a1017c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.252577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c285-13d4-4c21-9b2e-289187a1017c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.252851 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49c285-13d4-4c21-9b2e-289187a1017c admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.253502 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49c285-13d4-4c21-9b2e-289187a1017c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:12.256143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49c285-13d4-4c21-9b2e-289187a1017c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:12.282461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1386/2737] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:22:12 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:22:12.287016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:12.297597 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac1ebf-509a-4e40-9d9d-7c283af83781 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:12.366775 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18012438-186a-4f31-97cd-7d8bcf7f6d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:12.367461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18012438-186a-4f31-97cd-7d8bcf7f6d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.367637 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18012438-186a-4f31-97cd-7d8bcf7f6d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.367909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18012438-186a-4f31-97cd-7d8bcf7f6d94 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.368570 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18012438-186a-4f31-97cd-7d8bcf7f6d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:12.370464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18012438-186a-4f31-97cd-7d8bcf7f6d94 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:12.392730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1352/2738] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:22:12 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:22:12.397134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49c285-13d4-4c21-9b2e-289187a1017c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:12.407547 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49c285-13d4-4c21-9b2e-289187a1017c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:12.413071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35271c-1e0a-4f35-a1d2-9b3ba55753fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:12.413606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35271c-1e0a-4f35-a1d2-9b3ba55753fc admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.413887 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35271c-1e0a-4f35-a1d2-9b3ba55753fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.414163 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35271c-1e0a-4f35-a1d2-9b3ba55753fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.414801 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35271c-1e0a-4f35-a1d2-9b3ba55753fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:12.416461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35271c-1e0a-4f35-a1d2-9b3ba55753fc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:12.427407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1387/2739] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:22:12.431339 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18012438-186a-4f31-97cd-7d8bcf7f6d94 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:12.441191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18012438-186a-4f31-97cd-7d8bcf7f6d94 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:12.446157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2fe7-0ec4-4771-9b02-0e6972852841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:12.446744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2fe7-0ec4-4771-9b02-0e6972852841 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.447018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2fe7-0ec4-4771-9b02-0e6972852841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.447287 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2fe7-0ec4-4771-9b02-0e6972852841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.447922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2fe7-0ec4-4771-9b02-0e6972852841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:12.449526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2fe7-0ec4-4771-9b02-0e6972852841 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:12.451514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:12.463988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1353/2740] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:22:12.465130 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35271c-1e0a-4f35-a1d2-9b3ba55753fc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:12.476057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35271c-1e0a-4f35-a1d2-9b3ba55753fc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:12.480721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:12.481415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.481697 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.481974 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] PATH_INFO: `/v3/projects/ad05f96cb97442a8b4e65c0eed78a619/users/42918021b741445abba9da785157369a/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.498107 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad05f96cb97442a8b4e65c0eed78a619, user_id=42918021b741445abba9da785157369a, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:12.500808 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:12.513620 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1388/2741] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:22:12 2022] PUT /identity/v3/projects/ad05f96cb97442a8b4e65c0eed78a619/users/42918021b741445abba9da785157369a/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:22:12.583886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7e96b-b408-42ec-a440-00ef865e215b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.584168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7e96b-b408-42ec-a440-00ef865e215b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.584467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7e96b-b408-42ec-a440-00ef865e215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.612487 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-caa7e96b-b408-42ec-a440-00ef865e215b None None] MFA Rules not processed for user `42918021b741445abba9da785157369a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:12.613721 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-caa7e96b-b408-42ec-a440-00ef865e215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:12.677921 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1354/2742] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:22:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:12.688613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:12.690304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:12.771614 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0ccb2-3053-4204-b24e-b86a799bb999 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:12.775282 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dd0e3-9b81-4a23-af61-d1894546e42e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:12.775876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dd0e3-9b81-4a23-af61-d1894546e42e service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:12.776150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dd0e3-9b81-4a23-af61-d1894546e42e service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:12.776417 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dd0e3-9b81-4a23-af61-d1894546e42e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:12.784590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dd0e3-9b81-4a23-af61-d1894546e42e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:12.786201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dd0e3-9b81-4a23-af61-d1894546e42e service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:12.795579 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1389/2743] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:22:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:22:16.655137 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7e96b-b408-42ec-a440-00ef865e215b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:16.656563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-caa7e96b-b408-42ec-a440-00ef865e215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:16.668770 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7e96b-b408-42ec-a440-00ef865e215b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:16.672569 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:16.673165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:16.673449 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:16.673720 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:16.675391 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:16.734010 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:16.735430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:16.745217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1355/2744] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:22:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:22:22.791437 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=67,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:23.669191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465890-39fd-4233-901f-3d6e76201550 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:23.669475 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465890-39fd-4233-901f-3d6e76201550 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:23.669749 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465890-39fd-4233-901f-3d6e76201550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:23.689758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a2465890-39fd-4233-901f-3d6e76201550 None None] MFA Rules not processed for user `f12cfb692b4b4947bc67a3199aa33e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:23.690957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a2465890-39fd-4233-901f-3d6e76201550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:23.698058 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1390/2745] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:22:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:22:24.966373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:25.032539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2d73a-04e2-47ff-9990-6b9cab9a01e3 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:25.038184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:25.038840 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:25.039278 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:25.040098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] PATH_INFO: `/v3/users/42918021b741445abba9da785157369a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:25.041512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42918021b741445abba9da785157369a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:25.043604 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:25.074261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:25.096028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42918021b741445abba9da785157369a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.104084 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42918021b741445abba9da785157369a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.111526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42918021b741445abba9da785157369a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.112159 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42918021b741445abba9da785157369a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.112785 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42918021b741445abba9da785157369a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.130392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1356/2746] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:22:24 2022] DELETE /identity/v3/users/42918021b741445abba9da785157369a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:22:25.134839 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d93b5-dbbd-47be-9e46-39041fb24319 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:25.135315 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d93b5-dbbd-47be-9e46-39041fb24319 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:25.135441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d93b5-dbbd-47be-9e46-39041fb24319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:25.153995 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d54d93b5-dbbd-47be-9e46-39041fb24319 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:25.155171 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d54d93b5-dbbd-47be-9e46-39041fb24319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:25.213555 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1391/2747] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:22:25.225980 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:25.227411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:25.285106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78083b28-91b2-42e0-84f5-32ba6a033ab4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:25.288707 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3471196-f718-4d60-b85c-91da83fe486d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:25.289306 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3471196-f718-4d60-b85c-91da83fe486d service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:25.290118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3471196-f718-4d60-b85c-91da83fe486d service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:25.290553 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3471196-f718-4d60-b85c-91da83fe486d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:25.299990 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3471196-f718-4d60-b85c-91da83fe486d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:25.301435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3471196-f718-4d60-b85c-91da83fe486d service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:25.312674 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1357/2748] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:22:25.621840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d93b5-dbbd-47be-9e46-39041fb24319 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:22:25.623452 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d54d93b5-dbbd-47be-9e46-39041fb24319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:25.673457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d93b5-dbbd-47be-9e46-39041fb24319 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:22:25.680646 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:22:25.681728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:25.682208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:25.683081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] PATH_INFO: `/v3/projects/ad05f96cb97442a8b4e65c0eed78a619` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:25.684287 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad05f96cb97442a8b4e65c0eed78a619)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:22:25.686319 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:22:25.735218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad05f96cb97442a8b4e65c0eed78a619', 'request_id': 'req-a66d3949-d710-4285-ab02-a71a5dc03eef'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.740993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad05f96cb97442a8b4e65c0eed78a619', 'request_id': 'req-a66d3949-d710-4285-ab02-a71a5dc03eef'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.750114 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad05f96cb97442a8b4e65c0eed78a619', 'request_id': 'req-a66d3949-d710-4285-ab02-a71a5dc03eef'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.762653 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] The token cache is being invalidate because project ad05f96cb97442a8b4e65c0eed78a619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:22:25.763017 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.763740 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:22:25.765441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1392/2749] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:22:25 2022] DELETE /identity/v3/projects/ad05f96cb97442a8b4e65c0eed78a619 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:22:35.307699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=45,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:35.759010 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:22:56.316524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6c61d-b2ae-4fcd-8adb-4fc0d0afda86 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:22:56.317052 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6c61d-b2ae-4fcd-8adb-4fc0d0afda86 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:22:56.317256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6c61d-b2ae-4fcd-8adb-4fc0d0afda86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:22:56.336902 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e7b6c61d-b2ae-4fcd-8adb-4fc0d0afda86 None None] MFA Rules not processed for user `4f7899ab15b347b7be75e3433f0d5450`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:22:56.338094 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e7b6c61d-b2ae-4fcd-8adb-4fc0d0afda86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:22:56.391806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1358/2750] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:22:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:23:06.384837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=25 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:23:54.597920 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:23:54.599323 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:23:54.656200 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3949-d710-4285-ab02-a71a5dc03eef admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:23:54.659909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:23:54.660494 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:23:54.660778 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:23:54.661053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:23:54.662816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:23:54.743998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:23:54.745580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:23:54.766680 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1393/2751] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:23:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:23:55.950682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6c61d-b2ae-4fcd-8adb-4fc0d0afda86 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:23:55.952177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e7b6c61d-b2ae-4fcd-8adb-4fc0d0afda86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:23:55.964425 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6c61d-b2ae-4fcd-8adb-4fc0d0afda86 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:23:55.968028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:23:55.968572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:23:55.968846 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:23:55.969113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:23:55.970753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:23:55.984533 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:23:55.985942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:23:55.996714 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1359/2752] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:23:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:04.764091 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=63 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:05.866088 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:05.868149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:05.936757 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa1dea-f52b-4731-a7e7-8ab341a2da53 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:05.941268 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:05.941909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:05.942187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:05.942463 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] PATH_INFO: `/v3/users/4f7899ab15b347b7be75e3433f0d5450` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:05.943602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7899ab15b347b7be75e3433f0d5450)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:05.945580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:05.991874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:05.999919 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7899ab15b347b7be75e3433f0d5450'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.007178 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7899ab15b347b7be75e3433f0d5450'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.015657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7899ab15b347b7be75e3433f0d5450'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.016353 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7899ab15b347b7be75e3433f0d5450'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.017020 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b643503a-34e7-4839-8cfb-32b33db44f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7899ab15b347b7be75e3433f0d5450'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.036522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1394/2753] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:05 2022] DELETE /identity/v3/users/4f7899ab15b347b7be75e3433f0d5450 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:06.224385 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:06.236413 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7892e8-20f0-4666-a6e1-a46e88ccb455 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:06.296526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:06.297078 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.297352 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.297618 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] PATH_INFO: `/v3/projects/46afad886ad34608b533f4ee80a18ba5` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.298796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46afad886ad34608b533f4ee80a18ba5)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:06.300742 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:06.346153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46afad886ad34608b533f4ee80a18ba5', 'request_id': 'req-005ccdb7-6da8-4a29-8e33-a6149a81aef0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.351445 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46afad886ad34608b533f4ee80a18ba5', 'request_id': 'req-005ccdb7-6da8-4a29-8e33-a6149a81aef0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.359100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46afad886ad34608b533f4ee80a18ba5', 'request_id': 'req-005ccdb7-6da8-4a29-8e33-a6149a81aef0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.369884 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] The token cache is being invalidate because project 46afad886ad34608b533f4ee80a18ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:06.370169 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.370844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-005ccdb7-6da8-4a29-8e33-a6149a81aef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:06.372266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1360/2754] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:06 2022] DELETE /identity/v3/projects/46afad886ad34608b533f4ee80a18ba5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:06.377371 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee86dab-f0fc-4816-9c24-8ef8c452788f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.377659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee86dab-f0fc-4816-9c24-8ef8c452788f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.377937 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee86dab-f0fc-4816-9c24-8ef8c452788f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.398158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0ee86dab-f0fc-4816-9c24-8ef8c452788f None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:06.399352 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0ee86dab-f0fc-4816-9c24-8ef8c452788f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:06.462418 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1395/2755] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:06.471179 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89acb5-8831-4e2c-8f2c-777c48692974 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.471501 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89acb5-8831-4e2c-8f2c-777c48692974 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.471778 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89acb5-8831-4e2c-8f2c-777c48692974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.493079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-df89acb5-8831-4e2c-8f2c-777c48692974 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:06.494289 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-df89acb5-8831-4e2c-8f2c-777c48692974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:06.502784 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1361/2756] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:06.507102 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee86dab-f0fc-4816-9c24-8ef8c452788f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:06.517777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee86dab-f0fc-4816-9c24-8ef8c452788f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:06.522560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:06.522885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.523194 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.523467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.524111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:24:06.524415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:06.526036 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:06.537602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1396/2757] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:24:06.542171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89acb5-8831-4e2c-8f2c-777c48692974 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:06.551582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89acb5-8831-4e2c-8f2c-777c48692974 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:06.558514 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbfba8-acd6-43ce-bfd9-7dcc443a8dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:06.559171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbfba8-acd6-43ce-bfd9-7dcc443a8dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.559444 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbfba8-acd6-43ce-bfd9-7dcc443a8dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.559712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbfba8-acd6-43ce-bfd9-7dcc443a8dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.560325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbfba8-acd6-43ce-bfd9-7dcc443a8dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:06.562212 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbfba8-acd6-43ce-bfd9-7dcc443a8dcf admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:06.598568 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1362/2758] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:06 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:06.603025 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:06.613503 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b690-4ca2-4c8b-817e-3bddd6acbbbd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:06.617039 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:06.672621 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6436-10a8-4c90-8378-f3870c182893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:06.673228 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6436-10a8-4c90-8378-f3870c182893 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.673506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6436-10a8-4c90-8378-f3870c182893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.673777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da6436-10a8-4c90-8378-f3870c182893 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.674403 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da6436-10a8-4c90-8378-f3870c182893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:06.676344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da6436-10a8-4c90-8378-f3870c182893 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:06.696919 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1397/2759] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:06 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:06.701171 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbfba8-acd6-43ce-bfd9-7dcc443a8dcf admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:06.711682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbfba8-acd6-43ce-bfd9-7dcc443a8dcf admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:06.716380 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c1e69-b48b-4850-9abe-ce301ade3370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:06.717090 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1e69-b48b-4850-9abe-ce301ade3370 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.717363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1e69-b48b-4850-9abe-ce301ade3370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.717629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1e69-b48b-4850-9abe-ce301ade3370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.718265 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c1e69-b48b-4850-9abe-ce301ade3370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:06.719946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c1e69-b48b-4850-9abe-ce301ade3370 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:06.730806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1363/2760] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:06.734848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6436-10a8-4c90-8378-f3870c182893 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:06.744747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da6436-10a8-4c90-8378-f3870c182893 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:06.750187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32bb86-134d-42e9-adce-5807d72f9f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:06.750777 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bb86-134d-42e9-adce-5807d72f9f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.751056 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bb86-134d-42e9-adce-5807d72f9f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.751329 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bb86-134d-42e9-adce-5807d72f9f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.751968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32bb86-134d-42e9-adce-5807d72f9f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:06.753635 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32bb86-134d-42e9-adce-5807d72f9f95 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:06.766431 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1398/2761] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:06.770521 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c1e69-b48b-4850-9abe-ce301ade3370 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:06.780948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c1e69-b48b-4850-9abe-ce301ade3370 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:06.785497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:06.786132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.786409 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.786678 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] PATH_INFO: `/v3/projects/109e5d91f0584aa4b85dc27928ae5538/users/8b1c6021afa542d8b5548c4dab3ebcfa/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.801822 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=109e5d91f0584aa4b85dc27928ae5538, user_id=8b1c6021afa542d8b5548c4dab3ebcfa, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:06.804558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:06.817463 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1364/2762] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:06 2022] PUT /identity/v3/projects/109e5d91f0584aa4b85dc27928ae5538/users/8b1c6021afa542d8b5548c4dab3ebcfa/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:06.828983 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0252-bab3-4c2e-be38-9b185d62bef5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:06.829266 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0252-bab3-4c2e-be38-9b185d62bef5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:06.829538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0252-bab3-4c2e-be38-9b185d62bef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:06.857432 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-039c0252-bab3-4c2e-be38-9b185d62bef5 None None] MFA Rules not processed for user `8b1c6021afa542d8b5548c4dab3ebcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:06.858629 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-039c0252-bab3-4c2e-be38-9b185d62bef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:06.924129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1399/2763] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:06.935573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:06.937290 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:06.958213 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:07.013543 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d5269-2051-4969-9d66-67f9099cb463 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:07.019151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685865e-0e20-44c5-9153-f7720d83d9f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:07.019765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685865e-0e20-44c5-9153-f7720d83d9f5 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:07.020039 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685865e-0e20-44c5-9153-f7720d83d9f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:07.020310 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685865e-0e20-44c5-9153-f7720d83d9f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:07.028145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685865e-0e20-44c5-9153-f7720d83d9f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:07.029767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685865e-0e20-44c5-9153-f7720d83d9f5 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:07.038883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1365/2764] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:15.397596 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c0252-bab3-4c2e-be38-9b185d62bef5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:15.458311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c0252-bab3-4c2e-be38-9b185d62bef5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:15.462889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:15.463509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:15.463787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:15.464060 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] PATH_INFO: `/v3/users/8b1c6021afa542d8b5548c4dab3ebcfa` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:15.466103 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1c6021afa542d8b5548c4dab3ebcfa)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:15.468023 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:15.484279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:15.516499 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1c6021afa542d8b5548c4dab3ebcfa'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:15.523116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1c6021afa542d8b5548c4dab3ebcfa'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:15.530559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1c6021afa542d8b5548c4dab3ebcfa'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:15.531201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1c6021afa542d8b5548c4dab3ebcfa'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:15.531810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1c6021afa542d8b5548c4dab3ebcfa'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:15.549420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1400/2765] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:15 2022] DELETE /identity/v3/users/8b1c6021afa542d8b5548c4dab3ebcfa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:15.554011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1c750-8e80-4fec-a76b-cb48f06ccbf9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:15.554467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1c750-8e80-4fec-a76b-cb48f06ccbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:15.554688 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1c750-8e80-4fec-a76b-cb48f06ccbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:15.573535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-22d1c750-8e80-4fec-a76b-cb48f06ccbf9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:15.574805 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-22d1c750-8e80-4fec-a76b-cb48f06ccbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:15.632876 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1366/2766] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:15.643239 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:15.644660 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:15.701701 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a41197-4c6d-4087-92ee-17d7a2313073 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:15.705359 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d58f1-c79c-407c-b7b1-abf7853b9a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:15.705954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d58f1-c79c-407c-b7b1-abf7853b9a92 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:15.706234 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d58f1-c79c-407c-b7b1-abf7853b9a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:15.706505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d58f1-c79c-407c-b7b1-abf7853b9a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:15.716126 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d58f1-c79c-407c-b7b1-abf7853b9a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:15.717613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d58f1-c79c-407c-b7b1-abf7853b9a92 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:15.726988 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1401/2767] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:16.012907 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1c750-8e80-4fec-a76b-cb48f06ccbf9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.014403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-22d1c750-8e80-4fec-a76b-cb48f06ccbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:16.028883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1c750-8e80-4fec-a76b-cb48f06ccbf9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.033997 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.034595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.034873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.035151 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] PATH_INFO: `/v3/projects/109e5d91f0584aa4b85dc27928ae5538` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.036288 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=109e5d91f0584aa4b85dc27928ae5538)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.038564 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.090630 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '109e5d91f0584aa4b85dc27928ae5538', 'request_id': 'req-9db88e62-5f48-4789-86cd-856a43a0a603'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:16.096149 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '109e5d91f0584aa4b85dc27928ae5538', 'request_id': 'req-9db88e62-5f48-4789-86cd-856a43a0a603'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:16.107260 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '109e5d91f0584aa4b85dc27928ae5538', 'request_id': 'req-9db88e62-5f48-4789-86cd-856a43a0a603'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:16.120116 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] The token cache is being invalidate because project 109e5d91f0584aa4b85dc27928ae5538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:16.120418 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:16.121765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9db88e62-5f48-4789-86cd-856a43a0a603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:16.123652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1367/2768] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:16 2022] DELETE /identity/v3/projects/109e5d91f0584aa4b85dc27928ae5538 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:16.128805 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b9103-7c39-4e4e-9921-d8e1a79111c2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.129089 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b9103-7c39-4e4e-9921-d8e1a79111c2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.129369 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b9103-7c39-4e4e-9921-d8e1a79111c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.149738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-513b9103-7c39-4e4e-9921-d8e1a79111c2 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:16.150886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-513b9103-7c39-4e4e-9921-d8e1a79111c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:16.217221 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1402/2769] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:16.225381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab609-e7ee-46b9-9ffb-885dce68a24b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.225571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab609-e7ee-46b9-9ffb-885dce68a24b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.225860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab609-e7ee-46b9-9ffb-885dce68a24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.247485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-903ab609-e7ee-46b9-9ffb-885dce68a24b None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:16.248723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-903ab609-e7ee-46b9-9ffb-885dce68a24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:16.256757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1368/2770] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:24:16.261290 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b9103-7c39-4e4e-9921-d8e1a79111c2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.270998 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b9103-7c39-4e4e-9921-d8e1a79111c2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.281454 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.282063 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.282339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.282609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.283260 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:24:16.283561 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.285227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.297663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1403/2771] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:24:16.302145 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ab609-e7ee-46b9-9ffb-885dce68a24b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.311392 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ab609-e7ee-46b9-9ffb-885dce68a24b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.316119 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f897d4f-0f98-4d16-afa9-5ee6a77fe9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.316750 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897d4f-0f98-4d16-afa9-5ee6a77fe9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.317030 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897d4f-0f98-4d16-afa9-5ee6a77fe9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.317332 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897d4f-0f98-4d16-afa9-5ee6a77fe9a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.317964 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f897d4f-0f98-4d16-afa9-5ee6a77fe9a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.319882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f897d4f-0f98-4d16-afa9-5ee6a77fe9a1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.340783 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:16.352600 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1369/2772] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:16 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:16.353987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.364647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb7cc2-5fc1-4b3b-a702-b055f12e6a00 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.385552 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:16.424081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98aba-6271-4a95-ab81-4734f12d96c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.424812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98aba-6271-4a95-ab81-4734f12d96c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.424976 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98aba-6271-4a95-ab81-4734f12d96c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.425247 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98aba-6271-4a95-ab81-4734f12d96c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.425865 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b98aba-6271-4a95-ab81-4734f12d96c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.427780 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b98aba-6271-4a95-ab81-4734f12d96c5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.450612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1404/2773] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:16 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:16.454765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f897d4f-0f98-4d16-afa9-5ee6a77fe9a1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.465779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f897d4f-0f98-4d16-afa9-5ee6a77fe9a1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.470381 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ec69-e3d2-47c8-8426-bb03dea10ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.471016 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ec69-e3d2-47c8-8426-bb03dea10ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.471291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ec69-e3d2-47c8-8426-bb03dea10ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.471561 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ec69-e3d2-47c8-8426-bb03dea10ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.472316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18ec69-e3d2-47c8-8426-bb03dea10ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.473999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18ec69-e3d2-47c8-8426-bb03dea10ddb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.484368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1370/2774] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:16.488606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98aba-6271-4a95-ab81-4734f12d96c5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.498517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98aba-6271-4a95-ab81-4734f12d96c5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.503127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f036a-0522-41de-a804-65f47c7ff3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.503741 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f036a-0522-41de-a804-65f47c7ff3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.504019 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f036a-0522-41de-a804-65f47c7ff3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.504289 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f036a-0522-41de-a804-65f47c7ff3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.504973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f036a-0522-41de-a804-65f47c7ff3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.506644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f036a-0522-41de-a804-65f47c7ff3f6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.518367 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1405/2775] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:16.522968 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ec69-e3d2-47c8-8426-bb03dea10ddb admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.534274 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ec69-e3d2-47c8-8426-bb03dea10ddb admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.539147 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d755397-b144-4a7b-bce0-0c4e9f9c39fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.539802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d755397-b144-4a7b-bce0-0c4e9f9c39fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.540082 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d755397-b144-4a7b-bce0-0c4e9f9c39fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.540360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d755397-b144-4a7b-bce0-0c4e9f9c39fa admin admin] PATH_INFO: `/v3/projects/8fce4bfc192e49cbbfaa8c2e216c34cd/users/3279b363877149cf95c8d20d0aa01de1/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.556298 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d755397-b144-4a7b-bce0-0c4e9f9c39fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fce4bfc192e49cbbfaa8c2e216c34cd, user_id=3279b363877149cf95c8d20d0aa01de1, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.559080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d755397-b144-4a7b-bce0-0c4e9f9c39fa admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.573215 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1371/2776] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:16 2022] PUT /identity/v3/projects/8fce4bfc192e49cbbfaa8c2e216c34cd/users/3279b363877149cf95c8d20d0aa01de1/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:24:16.584139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca36032-d97b-4205-8943-25ac396749ff None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.584426 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca36032-d97b-4205-8943-25ac396749ff None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.584737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca36032-d97b-4205-8943-25ac396749ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.613558 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1ca36032-d97b-4205-8943-25ac396749ff None None] MFA Rules not processed for user `3279b363877149cf95c8d20d0aa01de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:16.614695 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1ca36032-d97b-4205-8943-25ac396749ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:16.677641 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1406/2777] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:16.682549 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d755397-b144-4a7b-bce0-0c4e9f9c39fa admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.741836 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d755397-b144-4a7b-bce0-0c4e9f9c39fa admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.746960 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cbba4-d99d-46c5-870e-c832e927e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.747555 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbba4-d99d-46c5-870e-c832e927e0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.747834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbba4-d99d-46c5-870e-c832e927e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.748111 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbba4-d99d-46c5-870e-c832e927e0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.748749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cbba4-d99d-46c5-870e-c832e927e0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.750622 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cbba4-d99d-46c5-870e-c832e927e0c7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.776526 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1372/2778] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:16 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:16.780667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca36032-d97b-4205-8943-25ac396749ff None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.791070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca36032-d97b-4205-8943-25ac396749ff None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.847202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce248b8-484e-45d1-ba9f-bfc2b60fd10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.847802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce248b8-484e-45d1-ba9f-bfc2b60fd10b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.848081 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce248b8-484e-45d1-ba9f-bfc2b60fd10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.848354 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce248b8-484e-45d1-ba9f-bfc2b60fd10b admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.849015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce248b8-484e-45d1-ba9f-bfc2b60fd10b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.851004 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce248b8-484e-45d1-ba9f-bfc2b60fd10b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.872067 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1407/2779] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:16 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:24:16.876203 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cbba4-d99d-46c5-870e-c832e927e0c7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.886347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cbba4-d99d-46c5-870e-c832e927e0c7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.892003 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3c79d-e74f-478b-8537-9ad6717cd609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.892631 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c79d-e74f-478b-8537-9ad6717cd609 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.892911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c79d-e74f-478b-8537-9ad6717cd609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.893186 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c79d-e74f-478b-8537-9ad6717cd609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.893827 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3c79d-e74f-478b-8537-9ad6717cd609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.895408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3c79d-e74f-478b-8537-9ad6717cd609 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.906390 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1373/2780] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:16.910458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce248b8-484e-45d1-ba9f-bfc2b60fd10b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.920506 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce248b8-484e-45d1-ba9f-bfc2b60fd10b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:16.924979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ae42-bc9d-4d49-add4-0ba2bab3b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:16.925613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ae42-bc9d-4d49-add4-0ba2bab3b98f admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:16.925886 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ae42-bc9d-4d49-add4-0ba2bab3b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:16.926158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ae42-bc9d-4d49-add4-0ba2bab3b98f admin admin] PATH_INFO: `/v3/projects/290ca5757f514a3aabd40c167f5dd1f4/users/894957df111e4be0909fad5be652273e/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:16.941279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13ae42-bc9d-4d49-add4-0ba2bab3b98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=290ca5757f514a3aabd40c167f5dd1f4, user_id=894957df111e4be0909fad5be652273e, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:16.943926 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13ae42-bc9d-4d49-add4-0ba2bab3b98f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:16.952991 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:16.959947 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1408/2781] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:16 2022] PUT /identity/v3/projects/290ca5757f514a3aabd40c167f5dd1f4/users/894957df111e4be0909fad5be652273e/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:16.963917 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3c79d-e74f-478b-8537-9ad6717cd609 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:16.973882 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3c79d-e74f-478b-8537-9ad6717cd609 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:17.031856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cd2b3-c326-4ad5-a0b8-890d99123a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:17.032492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd2b3-c326-4ad5-a0b8-890d99123a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:17.032776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd2b3-c326-4ad5-a0b8-890d99123a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:17.033048 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cd2b3-c326-4ad5-a0b8-890d99123a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:17.033692 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cd2b3-c326-4ad5-a0b8-890d99123a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:17.035269 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cd2b3-c326-4ad5-a0b8-890d99123a39 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:17.045335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1374/2782] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:17.049565 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ae42-bc9d-4d49-add4-0ba2bab3b98f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:17.060278 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ae42-bc9d-4d49-add4-0ba2bab3b98f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:17.064806 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:17.065441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:17.065719 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:17.066007 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] PATH_INFO: `/v3/projects/290ca5757f514a3aabd40c167f5dd1f4/users/894957df111e4be0909fad5be652273e/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:17.067882 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] RBAC: Authorizing `identity:create_grant(project_id=290ca5757f514a3aabd40c167f5dd1f4, user_id=894957df111e4be0909fad5be652273e, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:17.070537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:17.081957 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1409/2783] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:17 2022] PUT /identity/v3/projects/290ca5757f514a3aabd40c167f5dd1f4/users/894957df111e4be0909fad5be652273e/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:17.093534 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd28fdf-c4f2-44ed-91f0-789c023fc5dd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:17.093815 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd28fdf-c4f2-44ed-91f0-789c023fc5dd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:17.094085 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd28fdf-c4f2-44ed-91f0-789c023fc5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:17.122875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-cbd28fdf-c4f2-44ed-91f0-789c023fc5dd None None] MFA Rules not processed for user `894957df111e4be0909fad5be652273e`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:17.123992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cbd28fdf-c4f2-44ed-91f0-789c023fc5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:17.143613 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:17.196362 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1375/2784] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:17.208059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:17.209786 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:17.278456 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11916d93-caf5-4355-bc79-3a635da1215f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:17.282335 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea6a16-b062-4726-a50f-de353ee90431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:17.283051 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea6a16-b062-4726-a50f-de353ee90431 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:17.283222 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea6a16-b062-4726-a50f-de353ee90431 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:17.283727 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea6a16-b062-4726-a50f-de353ee90431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:17.293710 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea6a16-b062-4726-a50f-de353ee90431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:17.295146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea6a16-b062-4726-a50f-de353ee90431 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:17.306330 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1410/2785] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:17.719123 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd28fdf-c4f2-44ed-91f0-789c023fc5dd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:17.785699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd28fdf-c4f2-44ed-91f0-789c023fc5dd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:17.790255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:17.790887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:17.791161 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:17.791430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] PATH_INFO: `/v3/users/3279b363877149cf95c8d20d0aa01de1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:17.792626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3279b363877149cf95c8d20d0aa01de1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:17.795202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:17.844008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3279b363877149cf95c8d20d0aa01de1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:17.851680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3279b363877149cf95c8d20d0aa01de1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:17.859320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3279b363877149cf95c8d20d0aa01de1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:17.860087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3279b363877149cf95c8d20d0aa01de1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:17.860811 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3279b363877149cf95c8d20d0aa01de1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:17.880100 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1376/2786] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:17 2022] DELETE /identity/v3/users/3279b363877149cf95c8d20d0aa01de1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:17.885618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9398af5-c7a3-4f0a-99a7-165a54cd538a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:17.886119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9398af5-c7a3-4f0a-99a7-165a54cd538a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:17.886358 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9398af5-c7a3-4f0a-99a7-165a54cd538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:17.909141 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a9398af5-c7a3-4f0a-99a7-165a54cd538a None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:17.910350 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a9398af5-c7a3-4f0a-99a7-165a54cd538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:17.974486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1411/2787] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:17.985214 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:17.986626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:18.031668 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:18.049473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce043b-8251-4202-a9d7-a8a33a9d761e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:18.053154 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36a406-1b35-4a89-b86f-ce1a26aeb5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:18.053789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36a406-1b35-4a89-b86f-ce1a26aeb5eb service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:18.054067 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36a406-1b35-4a89-b86f-ce1a26aeb5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:18.054335 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36a406-1b35-4a89-b86f-ce1a26aeb5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:18.063903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36a406-1b35-4a89-b86f-ce1a26aeb5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:18.065353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36a406-1b35-4a89-b86f-ce1a26aeb5eb service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:18.075925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1377/2788] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:18.429737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9398af5-c7a3-4f0a-99a7-165a54cd538a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:18.431206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a9398af5-c7a3-4f0a-99a7-165a54cd538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:18.443931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9398af5-c7a3-4f0a-99a7-165a54cd538a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:18.449586 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:18.450265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:18.450537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:18.450810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] PATH_INFO: `/v3/projects/8fce4bfc192e49cbbfaa8c2e216c34cd` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:18.454600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fce4bfc192e49cbbfaa8c2e216c34cd)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:18.457768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:18.572507 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fce4bfc192e49cbbfaa8c2e216c34cd', 'request_id': 'req-773cc806-7967-4cf7-9aed-f6f746e85c16'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.579505 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fce4bfc192e49cbbfaa8c2e216c34cd', 'request_id': 'req-773cc806-7967-4cf7-9aed-f6f746e85c16'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.588788 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fce4bfc192e49cbbfaa8c2e216c34cd', 'request_id': 'req-773cc806-7967-4cf7-9aed-f6f746e85c16'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.601477 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] The token cache is being invalidate because project 8fce4bfc192e49cbbfaa8c2e216c34cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:18.601766 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.602537 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.607434 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1412/2789] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:18 2022] DELETE /identity/v3/projects/8fce4bfc192e49cbbfaa8c2e216c34cd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:18.608725 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36a406-1b35-4a89-b86f-ce1a26aeb5eb service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:18.618258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36a406-1b35-4a89-b86f-ce1a26aeb5eb service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:18.676559 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:18.677180 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:18.677454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:18.677722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] PATH_INFO: `/v3/users/894957df111e4be0909fad5be652273e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:18.679018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=894957df111e4be0909fad5be652273e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:18.680935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:18.731166 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894957df111e4be0909fad5be652273e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.739397 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894957df111e4be0909fad5be652273e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.746518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894957df111e4be0909fad5be652273e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.747153 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894957df111e4be0909fad5be652273e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.747759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d350740-f94b-4fc3-9590-31e69bb1a8e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894957df111e4be0909fad5be652273e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:18.765497 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1378/2790] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:18 2022] DELETE /identity/v3/users/894957df111e4be0909fad5be652273e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:19.063129 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.064732 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:19.080744 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cc806-7967-4cf7-9aed-f6f746e85c16 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:19.090811 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:19.146453 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:19.147112 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.147389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.147662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] PATH_INFO: `/v3/projects/290ca5757f514a3aabd40c167f5dd1f4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.148814 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=290ca5757f514a3aabd40c167f5dd1f4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:19.150728 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:19.205739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '290ca5757f514a3aabd40c167f5dd1f4', 'request_id': 'req-b2fab56a-1721-4a4f-a935-2676da5f7ae4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:19.211690 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '290ca5757f514a3aabd40c167f5dd1f4', 'request_id': 'req-b2fab56a-1721-4a4f-a935-2676da5f7ae4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:19.223605 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '290ca5757f514a3aabd40c167f5dd1f4', 'request_id': 'req-b2fab56a-1721-4a4f-a935-2676da5f7ae4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:19.236002 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] The token cache is being invalidate because project 290ca5757f514a3aabd40c167f5dd1f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:19.236295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:19.237057 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b2fab56a-1721-4a4f-a935-2676da5f7ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:19.238557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1413/2791] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:19 2022] DELETE /identity/v3/projects/290ca5757f514a3aabd40c167f5dd1f4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:19.243835 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f711055-c443-410c-8d4b-17371ac9cb27 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.244118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f711055-c443-410c-8d4b-17371ac9cb27 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.244388 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f711055-c443-410c-8d4b-17371ac9cb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.265927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3f711055-c443-410c-8d4b-17371ac9cb27 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:19.267144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3f711055-c443-410c-8d4b-17371ac9cb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:19.319903 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:19.330194 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1379/2792] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:24:19.337735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e4c4-cc9b-45d4-9b35-16c43a1c7cb3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.338021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e4c4-cc9b-45d4-9b35-16c43a1c7cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.338295 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e4c4-cc9b-45d4-9b35-16c43a1c7cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.358217 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2570e4c4-cc9b-45d4-9b35-16c43a1c7cb3 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:19.359458 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2570e4c4-cc9b-45d4-9b35-16c43a1c7cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:19.367356 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1414/2793] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:24:19.372354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f711055-c443-410c-8d4b-17371ac9cb27 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.381860 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f711055-c443-410c-8d4b-17371ac9cb27 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:19.386464 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:19.387079 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.387337 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.387606 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.388342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:24:19.388711 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:19.390232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:19.401831 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1380/2794] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:24:19.406233 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e4c4-cc9b-45d4-9b35-16c43a1c7cb3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.415486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e4c4-cc9b-45d4-9b35-16c43a1c7cb3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:19.421275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df449e9a-b891-4097-94d2-d360086b2d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:19.421858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df449e9a-b891-4097-94d2-d360086b2d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.422137 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df449e9a-b891-4097-94d2-d360086b2d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.422409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df449e9a-b891-4097-94d2-d360086b2d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.423033 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df449e9a-b891-4097-94d2-d360086b2d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:19.424956 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df449e9a-b891-4097-94d2-d360086b2d1f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:19.455071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1415/2795] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:19 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:19.459173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.471321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef8e3e-9d08-4b07-8f19-895b457c0a7c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:19.529087 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e987b7d-c3ff-46b3-9c99-ffffb673f40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:19.529699 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e987b7d-c3ff-46b3-9c99-ffffb673f40f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.529988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e987b7d-c3ff-46b3-9c99-ffffb673f40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.530261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e987b7d-c3ff-46b3-9c99-ffffb673f40f admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.530898 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e987b7d-c3ff-46b3-9c99-ffffb673f40f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:19.532843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e987b7d-c3ff-46b3-9c99-ffffb673f40f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:19.557062 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1381/2796] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:19 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:24:19.563802 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df449e9a-b891-4097-94d2-d360086b2d1f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.575286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df449e9a-b891-4097-94d2-d360086b2d1f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:19.580510 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb28029-c6f0-4bfb-8be1-73eef0853bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:19.581116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb28029-c6f0-4bfb-8be1-73eef0853bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.581396 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb28029-c6f0-4bfb-8be1-73eef0853bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.581667 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb28029-c6f0-4bfb-8be1-73eef0853bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.582339 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb28029-c6f0-4bfb-8be1-73eef0853bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:19.585167 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb28029-c6f0-4bfb-8be1-73eef0853bd8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:19.596846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1416/2797] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:19.600936 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e987b7d-c3ff-46b3-9c99-ffffb673f40f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.611757 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e987b7d-c3ff-46b3-9c99-ffffb673f40f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:19.618184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c452e4-bff3-4ab2-a34d-7faf2dd7715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:19.618796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c452e4-bff3-4ab2-a34d-7faf2dd7715b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.619072 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c452e4-bff3-4ab2-a34d-7faf2dd7715b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.620034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c452e4-bff3-4ab2-a34d-7faf2dd7715b admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.620034 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c452e4-bff3-4ab2-a34d-7faf2dd7715b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:19.622404 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c452e4-bff3-4ab2-a34d-7faf2dd7715b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:19.635648 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1382/2798] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:19.640165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb28029-c6f0-4bfb-8be1-73eef0853bd8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.654944 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb28029-c6f0-4bfb-8be1-73eef0853bd8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:19.667306 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3642ab-efbd-446c-9227-1df4afee18de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:19.668921 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3642ab-efbd-446c-9227-1df4afee18de admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.669389 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3642ab-efbd-446c-9227-1df4afee18de admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.669673 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3642ab-efbd-446c-9227-1df4afee18de admin admin] PATH_INFO: `/v3/projects/e354a587ff874ff8a8eb2db0319c5098/users/1e9d633c673741998a4d149a79c38d68/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.688518 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3642ab-efbd-446c-9227-1df4afee18de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e354a587ff874ff8a8eb2db0319c5098, user_id=1e9d633c673741998a4d149a79c38d68, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:19.691205 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3642ab-efbd-446c-9227-1df4afee18de admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:19.707570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1417/2799] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:19 2022] PUT /identity/v3/projects/e354a587ff874ff8a8eb2db0319c5098/users/1e9d633c673741998a4d149a79c38d68/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:19.718196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736cb4f-ec58-4f5f-86b0-2f913fae8837 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.718571 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736cb4f-ec58-4f5f-86b0-2f913fae8837 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.718856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736cb4f-ec58-4f5f-86b0-2f913fae8837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.756893 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9736cb4f-ec58-4f5f-86b0-2f913fae8837 None None] MFA Rules not processed for user `1e9d633c673741998a4d149a79c38d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:19.758280 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9736cb4f-ec58-4f5f-86b0-2f913fae8837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:19.833006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1383/2800] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:19.837527 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3642ab-efbd-446c-9227-1df4afee18de admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.887799 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:19.913854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3642ab-efbd-446c-9227-1df4afee18de admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:19.919410 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ecc95-2e56-4add-ac84-d304be7e1062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:19.920013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ecc95-2e56-4add-ac84-d304be7e1062 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:19.920311 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ecc95-2e56-4add-ac84-d304be7e1062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:19.920609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ecc95-2e56-4add-ac84-d304be7e1062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:19.921236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ecc95-2e56-4add-ac84-d304be7e1062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:19.923125 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ecc95-2e56-4add-ac84-d304be7e1062 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:19.953392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1418/2801] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:19 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:24:19.957468 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736cb4f-ec58-4f5f-86b0-2f913fae8837 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:19.971193 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736cb4f-ec58-4f5f-86b0-2f913fae8837 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:20.023952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:20.033204 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3efd-6ed0-404b-b1d7-13431ed0a078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:20.033995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3efd-6ed0-404b-b1d7-13431ed0a078 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:20.034363 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3efd-6ed0-404b-b1d7-13431ed0a078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:20.034731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3efd-6ed0-404b-b1d7-13431ed0a078 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:20.035454 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c3efd-6ed0-404b-b1d7-13431ed0a078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:20.037558 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c3efd-6ed0-404b-b1d7-13431ed0a078 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:20.058680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1384/2802] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:19 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:20.062662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ecc95-2e56-4add-ac84-d304be7e1062 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:20.072987 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ecc95-2e56-4add-ac84-d304be7e1062 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:20.078032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d681ee-3349-4e95-ac69-40aeaa80bbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:20.078765 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d681ee-3349-4e95-ac69-40aeaa80bbd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:20.079138 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d681ee-3349-4e95-ac69-40aeaa80bbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:20.079549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d681ee-3349-4e95-ac69-40aeaa80bbd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:20.080375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d681ee-3349-4e95-ac69-40aeaa80bbd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:20.082111 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d681ee-3349-4e95-ac69-40aeaa80bbd4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:20.093885 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1419/2803] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:20.098058 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3efd-6ed0-404b-b1d7-13431ed0a078 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:20.108908 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3efd-6ed0-404b-b1d7-13431ed0a078 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:20.113612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae5e99-4f94-4b0c-bbb5-f4810ce56797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:20.114264 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae5e99-4f94-4b0c-bbb5-f4810ce56797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:20.114665 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae5e99-4f94-4b0c-bbb5-f4810ce56797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:20.115146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae5e99-4f94-4b0c-bbb5-f4810ce56797 admin admin] PATH_INFO: `/v3/projects/32486c929c9641b5b22cc5d303583ffa/users/35452653d80947cb9a6d281431370ed5/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:20.133060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae5e99-4f94-4b0c-bbb5-f4810ce56797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32486c929c9641b5b22cc5d303583ffa, user_id=35452653d80947cb9a6d281431370ed5, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:20.137875 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae5e99-4f94-4b0c-bbb5-f4810ce56797 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:20.151942 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1385/2804] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:20 2022] PUT /identity/v3/projects/32486c929c9641b5b22cc5d303583ffa/users/35452653d80947cb9a6d281431370ed5/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:20.156600 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d681ee-3349-4e95-ac69-40aeaa80bbd4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:20.167113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d681ee-3349-4e95-ac69-40aeaa80bbd4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:20.225411 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6b4d7-14c4-4e4e-b883-d4140350159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:20.225970 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6b4d7-14c4-4e4e-b883-d4140350159b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:20.226309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6b4d7-14c4-4e4e-b883-d4140350159b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:20.226550 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6b4d7-14c4-4e4e-b883-d4140350159b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:20.227269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6b4d7-14c4-4e4e-b883-d4140350159b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:20.228974 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6b4d7-14c4-4e4e-b883-d4140350159b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:20.239982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1420/2805] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:20.244091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae5e99-4f94-4b0c-bbb5-f4810ce56797 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:20.255254 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae5e99-4f94-4b0c-bbb5-f4810ce56797 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:20.259899 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:20.260701 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:20.260986 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:20.261263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] PATH_INFO: `/v3/projects/32486c929c9641b5b22cc5d303583ffa/users/35452653d80947cb9a6d281431370ed5/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:20.264102 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32486c929c9641b5b22cc5d303583ffa, user_id=35452653d80947cb9a6d281431370ed5, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:20.267069 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:20.278554 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1386/2806] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:20 2022] PUT /identity/v3/projects/32486c929c9641b5b22cc5d303583ffa/users/35452653d80947cb9a6d281431370ed5/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:20.346666 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254fb9-c79a-4eec-886e-32e6e8b2c471 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:20.347097 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254fb9-c79a-4eec-886e-32e6e8b2c471 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:20.347457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254fb9-c79a-4eec-886e-32e6e8b2c471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:20.379798 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-32254fb9-c79a-4eec-886e-32e6e8b2c471 None None] MFA Rules not processed for user `35452653d80947cb9a6d281431370ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:20.381387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-32254fb9-c79a-4eec-886e-32e6e8b2c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:20.464201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1421/2807] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:20.476315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:20.478133 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:20.539517 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50176761-0910-4189-ab86-2a5b101898b5 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:20.543730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1358be5-7c04-4a11-9c5a-dd852070bf32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:20.544354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1358be5-7c04-4a11-9c5a-dd852070bf32 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:20.544638 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1358be5-7c04-4a11-9c5a-dd852070bf32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:20.544912 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1358be5-7c04-4a11-9c5a-dd852070bf32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:20.553819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1358be5-7c04-4a11-9c5a-dd852070bf32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:20.555240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1358be5-7c04-4a11-9c5a-dd852070bf32 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:20.564680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1387/2808] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:21.036310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32254fb9-c79a-4eec-886e-32e6e8b2c471 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:21.080598 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:21.112927 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32254fb9-c79a-4eec-886e-32e6e8b2c471 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:21.118448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:21.119160 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:21.119435 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:21.119738 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] PATH_INFO: `/v3/users/1e9d633c673741998a4d149a79c38d68` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:21.120950 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e9d633c673741998a4d149a79c38d68)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:21.123374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:21.173612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e9d633c673741998a4d149a79c38d68'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.181238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e9d633c673741998a4d149a79c38d68'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.190467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e9d633c673741998a4d149a79c38d68'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.191215 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e9d633c673741998a4d149a79c38d68'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.191840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e9d633c673741998a4d149a79c38d68'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.215756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1422/2809] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:21 2022] DELETE /identity/v3/users/1e9d633c673741998a4d149a79c38d68 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:21.220172 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f3ec-5273-4aee-bcd4-548446a95b71 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:21.220753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f3ec-5273-4aee-bcd4-548446a95b71 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:21.220753 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f3ec-5273-4aee-bcd4-548446a95b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:21.241222 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0092f3ec-5273-4aee-bcd4-548446a95b71 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:21.242403 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0092f3ec-5273-4aee-bcd4-548446a95b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:21.311292 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1388/2810] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 19:24:21.320938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:21.322342 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:21.380657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988d02c-c604-4f62-b947-459892b3b2f6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:21.384675 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787db0a-5cd3-48eb-8cd9-5bcc48df5e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:21.385283 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787db0a-5cd3-48eb-8cd9-5bcc48df5e08 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:21.385556 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787db0a-5cd3-48eb-8cd9-5bcc48df5e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:21.385824 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787db0a-5cd3-48eb-8cd9-5bcc48df5e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:21.396627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787db0a-5cd3-48eb-8cd9-5bcc48df5e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:21.398064 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787db0a-5cd3-48eb-8cd9-5bcc48df5e08 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:21.408647 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1423/2811] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:21.690749 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092f3ec-5273-4aee-bcd4-548446a95b71 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:21.692173 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0092f3ec-5273-4aee-bcd4-548446a95b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:21.704419 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092f3ec-5273-4aee-bcd4-548446a95b71 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:21.708951 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:21.709582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:21.709857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:21.710122 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] PATH_INFO: `/v3/projects/e354a587ff874ff8a8eb2db0319c5098` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:21.711242 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e354a587ff874ff8a8eb2db0319c5098)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:21.713158 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:21.756563 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:21.759799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e354a587ff874ff8a8eb2db0319c5098', 'request_id': 'req-cfee0c4a-bcb5-4529-a643-501a99ce8525'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.771231 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e354a587ff874ff8a8eb2db0319c5098', 'request_id': 'req-cfee0c4a-bcb5-4529-a643-501a99ce8525'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.779915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e354a587ff874ff8a8eb2db0319c5098', 'request_id': 'req-cfee0c4a-bcb5-4529-a643-501a99ce8525'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.790806 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] The token cache is being invalidate because project e354a587ff874ff8a8eb2db0319c5098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:21.791091 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.791703 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.793649 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1389/2812] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:21 2022] DELETE /identity/v3/projects/e354a587ff874ff8a8eb2db0319c5098 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:21.797559 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787db0a-5cd3-48eb-8cd9-5bcc48df5e08 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:21.807085 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787db0a-5cd3-48eb-8cd9-5bcc48df5e08 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:21.856971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:21.864742 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:21.865421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:21.865860 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:21.866328 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] PATH_INFO: `/v3/users/35452653d80947cb9a6d281431370ed5` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:21.867539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35452653d80947cb9a6d281431370ed5)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:21.869604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:21.915035 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35452653d80947cb9a6d281431370ed5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.922146 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35452653d80947cb9a6d281431370ed5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.929674 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35452653d80947cb9a6d281431370ed5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.930959 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35452653d80947cb9a6d281431370ed5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.931774 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1693129c-6580-4358-b1cd-a3dcd96b1720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35452653d80947cb9a6d281431370ed5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:21.950591 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1424/2813] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:21 2022] DELETE /identity/v3/users/35452653d80947cb9a6d281431370ed5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:22.258802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:22.260864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:22.276243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee0c4a-bcb5-4529-a643-501a99ce8525 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:22.341786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:22.342382 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.342728 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.343007 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] PATH_INFO: `/v3/projects/32486c929c9641b5b22cc5d303583ffa` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.344176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32486c929c9641b5b22cc5d303583ffa)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:22.346146 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:22.410985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32486c929c9641b5b22cc5d303583ffa', 'request_id': 'req-18dca050-df86-4184-b3a5-40395a9a7f00'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:22.417344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32486c929c9641b5b22cc5d303583ffa', 'request_id': 'req-18dca050-df86-4184-b3a5-40395a9a7f00'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:22.428593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32486c929c9641b5b22cc5d303583ffa', 'request_id': 'req-18dca050-df86-4184-b3a5-40395a9a7f00'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:22.442909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] The token cache is being invalidate because project 32486c929c9641b5b22cc5d303583ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:22.443202 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:22.443984 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-18dca050-df86-4184-b3a5-40395a9a7f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:22.445530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1390/2814] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:22 2022] DELETE /identity/v3/projects/32486c929c9641b5b22cc5d303583ffa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:24:22.450996 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715baf54-cd84-4189-ad1c-a15ab45fcf19 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.451609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715baf54-cd84-4189-ad1c-a15ab45fcf19 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.451609 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715baf54-cd84-4189-ad1c-a15ab45fcf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.473069 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-715baf54-cd84-4189-ad1c-a15ab45fcf19 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:22.474321 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-715baf54-cd84-4189-ad1c-a15ab45fcf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:22.545652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1425/2815] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:22.553575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942099b7-c97d-40d8-a3d1-d3e7f64dce09 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.553857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942099b7-c97d-40d8-a3d1-d3e7f64dce09 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.554128 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942099b7-c97d-40d8-a3d1-d3e7f64dce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.574344 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-942099b7-c97d-40d8-a3d1-d3e7f64dce09 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:22.576182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-942099b7-c97d-40d8-a3d1-d3e7f64dce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:22.587550 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1391/2816] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:22.592535 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715baf54-cd84-4189-ad1c-a15ab45fcf19 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:22.602344 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715baf54-cd84-4189-ad1c-a15ab45fcf19 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:22.607076 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:22.607676 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.607954 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.608224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.608904 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:24:22.609203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:22.610816 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:22.623143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1426/2817] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 19:24:22.627291 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942099b7-c97d-40d8-a3d1-d3e7f64dce09 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:22.637572 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942099b7-c97d-40d8-a3d1-d3e7f64dce09 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:22.642313 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38ef4e-5321-4d2e-b4ba-56022d11d2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:22.642911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ef4e-5321-4d2e-b4ba-56022d11d2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.643217 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ef4e-5321-4d2e-b4ba-56022d11d2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.643489 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38ef4e-5321-4d2e-b4ba-56022d11d2b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.644106 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38ef4e-5321-4d2e-b4ba-56022d11d2b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:22.646033 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38ef4e-5321-4d2e-b4ba-56022d11d2b2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:22.677636 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1392/2818] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:22 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:22.681845 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:22.691626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c8b2b-07b8-4ba8-a8d3-095694f97bc5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:22.758244 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9e5f-bb53-4e47-80d4-edae8eb98fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:22.758868 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9e5f-bb53-4e47-80d4-edae8eb98fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.759139 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9e5f-bb53-4e47-80d4-edae8eb98fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.759414 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9e5f-bb53-4e47-80d4-edae8eb98fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.760038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9e5f-bb53-4e47-80d4-edae8eb98fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:22.762462 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9e5f-bb53-4e47-80d4-edae8eb98fc5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:22.771366 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:22.785637 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1427/2819] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:22 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:24:22.789722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38ef4e-5321-4d2e-b4ba-56022d11d2b2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:22.800988 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38ef4e-5321-4d2e-b4ba-56022d11d2b2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:22.805764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c38164-4d4d-4e63-9215-76cba1aa1d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:22.806355 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c38164-4d4d-4e63-9215-76cba1aa1d1a admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.806626 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c38164-4d4d-4e63-9215-76cba1aa1d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.806891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c38164-4d4d-4e63-9215-76cba1aa1d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.807523 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c38164-4d4d-4e63-9215-76cba1aa1d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:22.809436 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c38164-4d4d-4e63-9215-76cba1aa1d1a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:22.819281 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1393/2820] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:22.823208 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9e5f-bb53-4e47-80d4-edae8eb98fc5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:22.833489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9e5f-bb53-4e47-80d4-edae8eb98fc5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:22.838016 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c99d0-02d1-4115-94c1-ea8d4aaa3861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:22.838597 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c99d0-02d1-4115-94c1-ea8d4aaa3861 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.838871 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c99d0-02d1-4115-94c1-ea8d4aaa3861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.839184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c99d0-02d1-4115-94c1-ea8d4aaa3861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.840791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c99d0-02d1-4115-94c1-ea8d4aaa3861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:22.842421 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c99d0-02d1-4115-94c1-ea8d4aaa3861 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:22.853094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1428/2821] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:22.858224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c38164-4d4d-4e63-9215-76cba1aa1d1a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:22.868184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c38164-4d4d-4e63-9215-76cba1aa1d1a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:22.873348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f30964-3811-4657-844b-4e1856611d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:22.873992 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f30964-3811-4657-844b-4e1856611d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.874272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f30964-3811-4657-844b-4e1856611d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.874546 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f30964-3811-4657-844b-4e1856611d1b admin admin] PATH_INFO: `/v3/projects/8333362c67884c51b3f497edbf9ee879/users/452d6e828b1a40808823b25570345e87/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.889603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f30964-3811-4657-844b-4e1856611d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8333362c67884c51b3f497edbf9ee879, user_id=452d6e828b1a40808823b25570345e87, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:22.892255 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f30964-3811-4657-844b-4e1856611d1b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:22.900196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:22.906263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1394/2822] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:22 2022] PUT /identity/v3/projects/8333362c67884c51b3f497edbf9ee879/users/452d6e828b1a40808823b25570345e87/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:22.918387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb4a8f-5ee4-4bee-a92f-53fe7ac31393 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:22.918750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb4a8f-5ee4-4bee-a92f-53fe7ac31393 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:22.919119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb4a8f-5ee4-4bee-a92f-53fe7ac31393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:22.948117 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-91bb4a8f-5ee4-4bee-a92f-53fe7ac31393 None None] MFA Rules not processed for user `452d6e828b1a40808823b25570345e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:22.949593 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-91bb4a8f-5ee4-4bee-a92f-53fe7ac31393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:23.019912 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1429/2823] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:23.024575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f30964-3811-4657-844b-4e1856611d1b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.085257 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f30964-3811-4657-844b-4e1856611d1b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:23.090350 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44251dc-4cc0-45af-8cea-6f7f3032d5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:23.091038 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44251dc-4cc0-45af-8cea-6f7f3032d5ef admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.091317 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44251dc-4cc0-45af-8cea-6f7f3032d5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.091579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44251dc-4cc0-45af-8cea-6f7f3032d5ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.092183 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44251dc-4cc0-45af-8cea-6f7f3032d5ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:23.094077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44251dc-4cc0-45af-8cea-6f7f3032d5ef admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:23.122157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1395/2824] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:23 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:23.126404 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb4a8f-5ee4-4bee-a92f-53fe7ac31393 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.137919 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb4a8f-5ee4-4bee-a92f-53fe7ac31393 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:23.193626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a2b06-4837-4018-8327-8595136af2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:23.194237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a2b06-4837-4018-8327-8595136af2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.194517 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a2b06-4837-4018-8327-8595136af2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.194820 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a2b06-4837-4018-8327-8595136af2b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.195455 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a2b06-4837-4018-8327-8595136af2b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:23.197457 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a2b06-4837-4018-8327-8595136af2b6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:23.219852 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1430/2825] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:23 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:23.223957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44251dc-4cc0-45af-8cea-6f7f3032d5ef admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.236340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44251dc-4cc0-45af-8cea-6f7f3032d5ef admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:23.240896 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f41fb-5220-4cd7-b4b4-550760d05843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:23.241489 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f41fb-5220-4cd7-b4b4-550760d05843 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.241764 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f41fb-5220-4cd7-b4b4-550760d05843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.242032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f41fb-5220-4cd7-b4b4-550760d05843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.242873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f41fb-5220-4cd7-b4b4-550760d05843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:23.244460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f41fb-5220-4cd7-b4b4-550760d05843 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:23.254910 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1396/2826] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:23.260072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a2b06-4837-4018-8327-8595136af2b6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.274175 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a2b06-4837-4018-8327-8595136af2b6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:23.282533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4a4ab-f933-41fc-8145-d6adf287bdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:23.283236 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4a4ab-f933-41fc-8145-d6adf287bdca admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.283513 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4a4ab-f933-41fc-8145-d6adf287bdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.285606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4a4ab-f933-41fc-8145-d6adf287bdca admin admin] PATH_INFO: `/v3/projects/428d4b17965f4ceb8ff3f5740e84b134/users/f64c2905d7d9455092092249178a78dc/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.301933 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4a4ab-f933-41fc-8145-d6adf287bdca admin admin] RBAC: Authorizing `identity:create_grant(project_id=428d4b17965f4ceb8ff3f5740e84b134, user_id=f64c2905d7d9455092092249178a78dc, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:23.304633 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4a4ab-f933-41fc-8145-d6adf287bdca admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:23.317689 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1431/2827] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:23 2022] PUT /identity/v3/projects/428d4b17965f4ceb8ff3f5740e84b134/users/f64c2905d7d9455092092249178a78dc/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:23.321672 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f41fb-5220-4cd7-b4b4-550760d05843 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.332080 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f41fb-5220-4cd7-b4b4-550760d05843 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:23.395293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d11f0-1469-4313-b9b3-c829a7f99b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:23.395913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d11f0-1469-4313-b9b3-c829a7f99b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.396198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d11f0-1469-4313-b9b3-c829a7f99b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.396498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d11f0-1469-4313-b9b3-c829a7f99b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.397141 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d11f0-1469-4313-b9b3-c829a7f99b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:23.398738 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d11f0-1469-4313-b9b3-c829a7f99b34 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:23.409895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1397/2828] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:23.414109 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4a4ab-f933-41fc-8145-d6adf287bdca admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.424464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4a4ab-f933-41fc-8145-d6adf287bdca admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:23.429048 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:23.429692 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.429968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.430237 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] PATH_INFO: `/v3/projects/428d4b17965f4ceb8ff3f5740e84b134/users/f64c2905d7d9455092092249178a78dc/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.432136 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=428d4b17965f4ceb8ff3f5740e84b134, user_id=f64c2905d7d9455092092249178a78dc, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:23.435164 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:23.446086 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1432/2829] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:23 2022] PUT /identity/v3/projects/428d4b17965f4ceb8ff3f5740e84b134/users/f64c2905d7d9455092092249178a78dc/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:23.456624 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8a632-acbc-4b19-a5d0-c16490838134 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.456911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8a632-acbc-4b19-a5d0-c16490838134 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.457182 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8a632-acbc-4b19-a5d0-c16490838134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.485485 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f4a8a632-acbc-4b19-a5d0-c16490838134 None None] MFA Rules not processed for user `f64c2905d7d9455092092249178a78dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:23.486718 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f4a8a632-acbc-4b19-a5d0-c16490838134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:23.547911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:23.559184 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1398/2830] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:23.570687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.572557 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:23.598962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:23.631316 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc801e-b3b5-4d0c-ac99-4fb488a9a1bf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:23.635258 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320389d-3634-4d49-a0dd-b46a6167a5ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:23.635872 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320389d-3634-4d49-a0dd-b46a6167a5ce service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.636149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320389d-3634-4d49-a0dd-b46a6167a5ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.637539 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320389d-3634-4d49-a0dd-b46a6167a5ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.646375 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320389d-3634-4d49-a0dd-b46a6167a5ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:23.647754 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320389d-3634-4d49-a0dd-b46a6167a5ce service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:23.657577 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1433/2831] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:23.725011 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8a632-acbc-4b19-a5d0-c16490838134 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.789915 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8a632-acbc-4b19-a5d0-c16490838134 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:23.795989 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:23.796791 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.796999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.797187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] PATH_INFO: `/v3/users/452d6e828b1a40808823b25570345e87` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.798340 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=452d6e828b1a40808823b25570345e87)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:23.800258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:23.849277 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452d6e828b1a40808823b25570345e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:23.860315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452d6e828b1a40808823b25570345e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:23.867296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452d6e828b1a40808823b25570345e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:23.867927 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452d6e828b1a40808823b25570345e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:23.868575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452d6e828b1a40808823b25570345e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:23.888315 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1399/2832] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:23 2022] DELETE /identity/v3/users/452d6e828b1a40808823b25570345e87 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:23.892981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c60943-0d13-4114-b357-17bf5d4b4925 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:23.893269 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c60943-0d13-4114-b357-17bf5d4b4925 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:23.893543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c60943-0d13-4114-b357-17bf5d4b4925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:23.913246 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a9c60943-0d13-4114-b357-17bf5d4b4925 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:23.914464 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a9c60943-0d13-4114-b357-17bf5d4b4925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:23.982444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1434/2833] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:23.992498 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:23.994022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:24.055110 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a39f6-a6c3-4c40-adc6-32489a74c558 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:24.059562 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:24.060187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:24.060484 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:24.060758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:24.070612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:24.072098 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:24.083002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1400/2834] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:24.295812 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c60943-0d13-4114-b357-17bf5d4b4925 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:24.306279 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c60943-0d13-4114-b357-17bf5d4b4925 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:24.310224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:24.310962 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:24.311425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:24.311791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:24.313649 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:24.326265 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:24.327729 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:24.337876 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1435/2835] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:24.473629 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:24.475411 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:24.489853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b72df9-55c8-4ca4-bb93-116181c7f240 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:24.494467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:24.495093 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:24.495374 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:24.495695 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] PATH_INFO: `/v3/projects/8333362c67884c51b3f497edbf9ee879` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:24.496853 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] RBAC: Authorizing `identity:delete_project(project_id=8333362c67884c51b3f497edbf9ee879)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:24.498759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:24.510365 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:24.562829 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8333362c67884c51b3f497edbf9ee879', 'request_id': 'req-44c13c86-4efc-407d-a804-0fb384a697af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.569192 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8333362c67884c51b3f497edbf9ee879', 'request_id': 'req-44c13c86-4efc-407d-a804-0fb384a697af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.578670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8333362c67884c51b3f497edbf9ee879', 'request_id': 'req-44c13c86-4efc-407d-a804-0fb384a697af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.594722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] The token cache is being invalidate because project 8333362c67884c51b3f497edbf9ee879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:24.595167 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.595948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.597834 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1401/2836] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:24 2022] DELETE /identity/v3/projects/8333362c67884c51b3f497edbf9ee879 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:24.601980 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:24.613650 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472811e9-b510-488f-bc9f-ff07281d70b3 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:24.677545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:24.678190 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:24.678981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:24.678981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] PATH_INFO: `/v3/users/f64c2905d7d9455092092249178a78dc` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:24.680070 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64c2905d7d9455092092249178a78dc)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:24.682031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:24.736394 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64c2905d7d9455092092249178a78dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.744545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:24.747687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64c2905d7d9455092092249178a78dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.758173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64c2905d7d9455092092249178a78dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.759218 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64c2905d7d9455092092249178a78dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.759908 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd7d87c5-f271-4702-bd8a-e939624fcb2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64c2905d7d9455092092249178a78dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:24.779216 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1436/2837] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:24 2022] DELETE /identity/v3/users/f64c2905d7d9455092092249178a78dc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:25.196378 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:25.197837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:25.211786 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c13c86-4efc-407d-a804-0fb384a697af admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:25.280115 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:25.281022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.281343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.281651 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] PATH_INFO: `/v3/projects/428d4b17965f4ceb8ff3f5740e84b134` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.282886 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=428d4b17965f4ceb8ff3f5740e84b134)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:25.285458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:25.340408 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428d4b17965f4ceb8ff3f5740e84b134', 'request_id': 'req-7cbf9f20-6496-408d-8b61-0990e29fa425'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:25.345993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428d4b17965f4ceb8ff3f5740e84b134', 'request_id': 'req-7cbf9f20-6496-408d-8b61-0990e29fa425'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:25.353939 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428d4b17965f4ceb8ff3f5740e84b134', 'request_id': 'req-7cbf9f20-6496-408d-8b61-0990e29fa425'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:25.367586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] The token cache is being invalidate because project 428d4b17965f4ceb8ff3f5740e84b134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:25.368353 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:25.368787 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7cbf9f20-6496-408d-8b61-0990e29fa425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:25.370435 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1402/2838] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:25 2022] DELETE /identity/v3/projects/428d4b17965f4ceb8ff3f5740e84b134 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:25.375897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca093f88-d8cc-462e-ad2f-91a7df41cbd8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.376184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca093f88-d8cc-462e-ad2f-91a7df41cbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.376478 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca093f88-d8cc-462e-ad2f-91a7df41cbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.403281 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ca093f88-d8cc-462e-ad2f-91a7df41cbd8 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:25.404662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ca093f88-d8cc-462e-ad2f-91a7df41cbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:25.474005 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1437/2839] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:25.482320 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dcd68-2478-4c92-bf23-efc6d345da72 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.482843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dcd68-2478-4c92-bf23-efc6d345da72 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.482843 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dcd68-2478-4c92-bf23-efc6d345da72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.504789 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-512dcd68-2478-4c92-bf23-efc6d345da72 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:25.506009 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-512dcd68-2478-4c92-bf23-efc6d345da72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:25.514360 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1403/2840] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:25.518626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca093f88-d8cc-462e-ad2f-91a7df41cbd8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:25.528971 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca093f88-d8cc-462e-ad2f-91a7df41cbd8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:25.533870 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:25.534486 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.534760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.535031 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.535699 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:24:25.536072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:25.537724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:25.555838 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1438/2841] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:24:25.560913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dcd68-2478-4c92-bf23-efc6d345da72 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:25.570356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dcd68-2478-4c92-bf23-efc6d345da72 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:25.575329 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c934ad2-c2b1-4fd9-aaa6-6d9d3fc8fdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:25.575929 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c934ad2-c2b1-4fd9-aaa6-6d9d3fc8fdee admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.576206 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c934ad2-c2b1-4fd9-aaa6-6d9d3fc8fdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.576504 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c934ad2-c2b1-4fd9-aaa6-6d9d3fc8fdee admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.577127 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c934ad2-c2b1-4fd9-aaa6-6d9d3fc8fdee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:25.579008 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c934ad2-c2b1-4fd9-aaa6-6d9d3fc8fdee admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:25.608044 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1404/2842] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:25 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:25.612272 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:25.623084 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d878c3-7364-4361-b685-cee5c8d228df admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:25.686859 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718c05d-cbeb-408f-a4cd-5f11f10cde8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:25.687460 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718c05d-cbeb-408f-a4cd-5f11f10cde8d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.687737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718c05d-cbeb-408f-a4cd-5f11f10cde8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.688008 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718c05d-cbeb-408f-a4cd-5f11f10cde8d admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.688668 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718c05d-cbeb-408f-a4cd-5f11f10cde8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:25.690582 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718c05d-cbeb-408f-a4cd-5f11f10cde8d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:25.715038 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1439/2843] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:25 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 19:24:25.716582 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c934ad2-c2b1-4fd9-aaa6-6d9d3fc8fdee admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:25.721974 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:25.728325 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c934ad2-c2b1-4fd9-aaa6-6d9d3fc8fdee admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:25.734722 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026506e3-c6d3-42a3-bc41-e631d064ceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:25.735330 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026506e3-c6d3-42a3-bc41-e631d064ceac admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.735614 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026506e3-c6d3-42a3-bc41-e631d064ceac admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.735887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026506e3-c6d3-42a3-bc41-e631d064ceac admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.736603 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026506e3-c6d3-42a3-bc41-e631d064ceac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:25.738197 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026506e3-c6d3-42a3-bc41-e631d064ceac admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:25.750250 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1405/2844] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:24:25.754333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718c05d-cbeb-408f-a4cd-5f11f10cde8d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:25.765516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718c05d-cbeb-408f-a4cd-5f11f10cde8d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:25.770705 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba41c6-2e81-4487-a2ed-f6258c258572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:25.771423 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba41c6-2e81-4487-a2ed-f6258c258572 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.772079 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba41c6-2e81-4487-a2ed-f6258c258572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.772207 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba41c6-2e81-4487-a2ed-f6258c258572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.772978 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba41c6-2e81-4487-a2ed-f6258c258572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:25.775065 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba41c6-2e81-4487-a2ed-f6258c258572 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:25.790098 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1440/2845] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:24:25.795266 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026506e3-c6d3-42a3-bc41-e631d064ceac admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:25.806545 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026506e3-c6d3-42a3-bc41-e631d064ceac admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:25.811296 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d93b-9ee3-4c67-a499-8a4cad9f86f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:25.812092 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d93b-9ee3-4c67-a499-8a4cad9f86f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.812574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d93b-9ee3-4c67-a499-8a4cad9f86f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.812952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d93b-9ee3-4c67-a499-8a4cad9f86f6 admin admin] PATH_INFO: `/v3/projects/4cad99459af4493eb0bfe13a4374abc3/users/f491349a5ba74b69b769b5750e880f3b/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.829006 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921d93b-9ee3-4c67-a499-8a4cad9f86f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cad99459af4493eb0bfe13a4374abc3, user_id=f491349a5ba74b69b769b5750e880f3b, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:25.831796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921d93b-9ee3-4c67-a499-8a4cad9f86f6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:25.845823 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1406/2846] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:25 2022] PUT /identity/v3/projects/4cad99459af4493eb0bfe13a4374abc3/users/f491349a5ba74b69b769b5750e880f3b/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:25.858877 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779308-f598-487e-a7e7-46d814c4f008 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:25.859304 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779308-f598-487e-a7e7-46d814c4f008 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:25.859657 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779308-f598-487e-a7e7-46d814c4f008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:25.886708 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-48779308-f598-487e-a7e7-46d814c4f008 None None] MFA Rules not processed for user `f491349a5ba74b69b769b5750e880f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:25.888013 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-48779308-f598-487e-a7e7-46d814c4f008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:25.917127 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:25.971116 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1441/2847] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:25.975004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d93b-9ee3-4c67-a499-8a4cad9f86f6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:26.055060 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d93b-9ee3-4c67-a499-8a4cad9f86f6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:26.060677 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:26.061621 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.062088 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.062467 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.063199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:26.065253 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:26.097895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1407/2848] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:25 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:26.101909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48779308-f598-487e-a7e7-46d814c4f008 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:26.114748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48779308-f598-487e-a7e7-46d814c4f008 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:26.180700 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b60cf4-8b10-4cc2-991b-c81555341b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:26.181326 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b60cf4-8b10-4cc2-991b-c81555341b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.181606 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b60cf4-8b10-4cc2-991b-c81555341b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.181881 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b60cf4-8b10-4cc2-991b-c81555341b58 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.182545 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b60cf4-8b10-4cc2-991b-c81555341b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:26.189341 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b60cf4-8b10-4cc2-991b-c81555341b58 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:26.193077 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:26.195552 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:26.225380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1442/2849] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:26 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:26.229500 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b60cf4-8b10-4cc2-991b-c81555341b58 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:26.248846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b60cf4-8b10-4cc2-991b-c81555341b58 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:26.255122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe6b2-ea99-4dcb-9755-4165a5759a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:26.255737 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe6b2-ea99-4dcb-9755-4165a5759a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.256024 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe6b2-ea99-4dcb-9755-4165a5759a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.256302 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe6b2-ea99-4dcb-9755-4165a5759a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.257149 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fe6b2-ea99-4dcb-9755-4165a5759a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:26.259235 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fe6b2-ea99-4dcb-9755-4165a5759a12 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:26.270105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ba0c3-3333-4ab4-b3d5-31a3a065f2db admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:26.270973 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1443/2850] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:26.274739 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe6b2-ea99-4dcb-9755-4165a5759a12 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:26.283299 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:26.284018 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.284405 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.284779 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.288334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:26.295858 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe6b2-ea99-4dcb-9755-4165a5759a12 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:26.302475 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:26.303427 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b950-cfc5-47bb-82b7-95b2dd37d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:26.304398 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:26.305000 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b950-cfc5-47bb-82b7-95b2dd37d374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.306333 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b950-cfc5-47bb-82b7-95b2dd37d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.306890 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b950-cfc5-47bb-82b7-95b2dd37d374 admin admin] PATH_INFO: `/v3/projects/93ca04d3eac9475b81806529652d6482/users/875e07126ca946d3b4694dc4552128ae/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.316999 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1408/2851] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:24:26.328662 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775b950-cfc5-47bb-82b7-95b2dd37d374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ca04d3eac9475b81806529652d6482, user_id=875e07126ca946d3b4694dc4552128ae, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:26.332270 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775b950-cfc5-47bb-82b7-95b2dd37d374 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:26.358575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1444/2852] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:26 2022] PUT /identity/v3/projects/93ca04d3eac9475b81806529652d6482/users/875e07126ca946d3b4694dc4552128ae/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:26.359066 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:26.371793 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b75ec9-4c60-4951-9f0f-3df2965a9ce4 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:26.451887 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:26.461891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3596944-dccc-4bcf-96ed-3ce94ec326ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:26.462652 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3596944-dccc-4bcf-96ed-3ce94ec326ea admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.463243 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3596944-dccc-4bcf-96ed-3ce94ec326ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.463538 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3596944-dccc-4bcf-96ed-3ce94ec326ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.465338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3596944-dccc-4bcf-96ed-3ce94ec326ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:26.467857 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3596944-dccc-4bcf-96ed-3ce94ec326ea admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:26.483937 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1409/2853] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:24:26.486014 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b950-cfc5-47bb-82b7-95b2dd37d374 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:26.501800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b950-cfc5-47bb-82b7-95b2dd37d374 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:26.516159 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:26.517888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.517888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.517888 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] PATH_INFO: `/v3/projects/93ca04d3eac9475b81806529652d6482/users/875e07126ca946d3b4694dc4552128ae/roles/38ea9e633951415f8b0c0c663e24e8b7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.521810 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ca04d3eac9475b81806529652d6482, user_id=875e07126ca946d3b4694dc4552128ae, role_id=38ea9e633951415f8b0c0c663e24e8b7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:26.525015 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:26.539273 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1445/2854] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:26 2022] PUT /identity/v3/projects/93ca04d3eac9475b81806529652d6482/users/875e07126ca946d3b4694dc4552128ae/roles/38ea9e633951415f8b0c0c663e24e8b7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:26.550946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62eb6-19db-4f75-9238-783ec23a88f8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.551844 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62eb6-19db-4f75-9238-783ec23a88f8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.552334 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62eb6-19db-4f75-9238-783ec23a88f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.596593 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-49a62eb6-19db-4f75-9238-783ec23a88f8 None None] MFA Rules not processed for user `875e07126ca946d3b4694dc4552128ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:26.597922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-49a62eb6-19db-4f75-9238-783ec23a88f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:26.713168 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1410/2855] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:26.725906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:26.727747 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:26.827837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f910d3-270c-4759-a987-785c76730bf3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:26.834095 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d865317-cf76-4c21-b693-b962156c1e5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:26.835441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d865317-cf76-4c21-b693-b962156c1e5d service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:26.835441 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d865317-cf76-4c21-b693-b962156c1e5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:26.835834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d865317-cf76-4c21-b693-b962156c1e5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:26.862106 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:26.909522 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d865317-cf76-4c21-b693-b962156c1e5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:26.911113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d865317-cf76-4c21-b693-b962156c1e5d service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:26.921979 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1446/2856] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:29.600586 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a62eb6-19db-4f75-9238-783ec23a88f8 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:29.686321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a62eb6-19db-4f75-9238-783ec23a88f8 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:29.691240 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:29.691758 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:29.692037 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:29.692316 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] PATH_INFO: `/v3/users/f491349a5ba74b69b769b5750e880f3b` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:29.693451 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f491349a5ba74b69b769b5750e880f3b)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:29.695430 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:29.769198 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f491349a5ba74b69b769b5750e880f3b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:29.780922 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f491349a5ba74b69b769b5750e880f3b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:29.792518 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f491349a5ba74b69b769b5750e880f3b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:29.794244 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f491349a5ba74b69b769b5750e880f3b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:29.797263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f491349a5ba74b69b769b5750e880f3b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:29.829130 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:29.846759 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1411/2857] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:29 2022] DELETE /identity/v3/users/f491349a5ba74b69b769b5750e880f3b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:29.854409 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce962908-95c4-44b1-8d29-22f39cf59521 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:29.855934 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce962908-95c4-44b1-8d29-22f39cf59521 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:29.856425 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce962908-95c4-44b1-8d29-22f39cf59521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:29.881834 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ce962908-95c4-44b1-8d29-22f39cf59521 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:29.884119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce962908-95c4-44b1-8d29-22f39cf59521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:29.957574 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:29.959744 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:29.998047 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1447/2858] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:30.032982 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce962908-95c4-44b1-8d29-22f39cf59521 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:30.033199 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f8dc-abe4-4a29-a1ac-a0252c7db278 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:30.038014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:30.038675 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:30.039002 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:30.039258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:30.041386 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:30.049925 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce962908-95c4-44b1-8d29-22f39cf59521 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:30.054181 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:30.055134 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:30.055658 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:30.055775 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:30.056059 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:30.056338 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:30.065361 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1412/2859] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:30.070602 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:30.072053 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:30.082750 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1448/2860] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:24:30.490579 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:30.500731 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8820d34-43cb-4390-a422-3e4067123319 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:30.506377 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:30.507283 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:30.507767 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:30.508286 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:30.510356 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:30.588914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:30.591118 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:30.602575 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1413/2861] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:30.782543 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:30.784756 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:30.890173 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebf26d-e689-44e0-b9fd-6ef38db15808 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:30.896148 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:30.897202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:30.897202 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:30.897644 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:30.899397 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:30.913967 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:30.915669 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:30.928347 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1449/2862] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:31.015541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:31.017157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:31.036218 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77c00d-e431-47cc-b108-e407952dab07 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:31.061985 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:31.062641 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:31.062914 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:31.063185 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] PATH_INFO: `/v3/projects/4cad99459af4493eb0bfe13a4374abc3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:31.064321 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cad99459af4493eb0bfe13a4374abc3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:31.067673 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:31.138014 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cad99459af4493eb0bfe13a4374abc3', 'request_id': 'req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.145656 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cad99459af4493eb0bfe13a4374abc3', 'request_id': 'req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.155950 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cad99459af4493eb0bfe13a4374abc3', 'request_id': 'req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.178925 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] The token cache is being invalidate because project 4cad99459af4493eb0bfe13a4374abc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:31.179227 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.180165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.182671 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1414/2863] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:31 2022] DELETE /identity/v3/projects/4cad99459af4493eb0bfe13a4374abc3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:31.187938 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:31.204214 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419f4d3-91ad-4587-b116-79ae98f80ff1 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:31.254386 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:31.281158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:31.281720 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:31.281999 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:31.282275 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] PATH_INFO: `/v3/users/875e07126ca946d3b4694dc4552128ae` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:31.283444 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=875e07126ca946d3b4694dc4552128ae)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:31.285791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:31.367203 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875e07126ca946d3b4694dc4552128ae'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.376575 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875e07126ca946d3b4694dc4552128ae'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.388124 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875e07126ca946d3b4694dc4552128ae'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.388975 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875e07126ca946d3b4694dc4552128ae'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.389640 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875e07126ca946d3b4694dc4552128ae'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:31.416538 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1450/2864] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:31 2022] DELETE /identity/v3/users/875e07126ca946d3b4694dc4552128ae => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:32.063969 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.065440 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.077909 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:32.122627 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.126018 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.178324 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5288eaa-4cf3-4f88-bb37-0bb185ed74c5 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.217320 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229c15-e0bf-43d3-affb-e92145f43237 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.222787 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.223489 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.223795 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.224071 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] PATH_INFO: `/v3/users/f12cfb692b4b4947bc67a3199aa33e73` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.225227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12cfb692b4b4947bc67a3199aa33e73)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.227473 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.247027 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.247874 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.248165 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.248473 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.250272 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.262548 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.264134 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.274476 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1415/2865] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:32.275211 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.276635 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.280613 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12cfb692b4b4947bc67a3199aa33e73'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.287906 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12cfb692b4b4947bc67a3199aa33e73'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.290162 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d718c72-0854-486c-8a79-66fcf092df33 service placement] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.291618 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:32.296221 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.296996 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.297182 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12cfb692b4b4947bc67a3199aa33e73'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.297322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.297542 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] PATH_INFO: `/v3/projects/93ca04d3eac9475b81806529652d6482` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.298776 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ca04d3eac9475b81806529652d6482)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.298931 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12cfb692b4b4947bc67a3199aa33e73'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.299617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9b753678-8b90-47f8-9df6-7c0780204dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12cfb692b4b4947bc67a3199aa33e73'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.300799 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.322762 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1451/2866] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:32 2022] DELETE /identity/v3/users/f12cfb692b4b4947bc67a3199aa33e73 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:24:32.350402 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ca04d3eac9475b81806529652d6482', 'request_id': 'req-e4673de5-f819-493e-a24d-e211f8e20d75'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.357273 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ca04d3eac9475b81806529652d6482', 'request_id': 'req-e4673de5-f819-493e-a24d-e211f8e20d75'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.365891 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ca04d3eac9475b81806529652d6482', 'request_id': 'req-e4673de5-f819-493e-a24d-e211f8e20d75'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.380730 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] The token cache is being invalidate because project 93ca04d3eac9475b81806529652d6482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:32.381019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.381682 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e4673de5-f819-493e-a24d-e211f8e20d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.384709 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1416/2867] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:32 2022] DELETE /identity/v3/projects/93ca04d3eac9475b81806529652d6482 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:32.389206 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ffa8-dfb2-4d12-a3cf-b855abfc1494 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.389580 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ffa8-dfb2-4d12-a3cf-b855abfc1494 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.389985 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7ffa8-dfb2-4d12-a3cf-b855abfc1494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.410313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-abd7ffa8-dfb2-4d12-a3cf-b855abfc1494 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:32.411514 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-abd7ffa8-dfb2-4d12-a3cf-b855abfc1494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.475393 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1452/2868] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:32.483732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0025f0-1fc0-415f-a288-453e732082d9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.484105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0025f0-1fc0-415f-a288-453e732082d9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.484515 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0025f0-1fc0-415f-a288-453e732082d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.506856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3d0025f0-1fc0-415f-a288-453e732082d9 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:32.508205 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3d0025f0-1fc0-415f-a288-453e732082d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.516491 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1417/2869] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:32.520656 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7ffa8-dfb2-4d12-a3cf-b855abfc1494 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.531897 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7ffa8-dfb2-4d12-a3cf-b855abfc1494 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.537309 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.537994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.538286 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.538549 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.539199 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 19:24:32.539501 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.541172 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.556011 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1453/2870] 10.222.0.70 () {58 vars in 1153 bytes} [Tue Aug 9 19:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 19:24:32.561032 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0025f0-1fc0-415f-a288-453e732082d9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.571387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0025f0-1fc0-415f-a288-453e732082d9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.574843 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.577539 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd0868-20ab-4c7c-b550-f3410ef598f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.578241 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0868-20ab-4c7c-b550-f3410ef598f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.578374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.578478 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0868-20ab-4c7c-b550-f3410ef598f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.578595 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd0868-20ab-4c7c-b550-f3410ef598f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.579263 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd0868-20ab-4c7c-b550-f3410ef598f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.581535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd0868-20ab-4c7c-b550-f3410ef598f4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.597196 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893eef35-635c-4f08-b54e-7f0b8f07eb70 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.602143 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:32.603848 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.604681 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.605516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.605840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] PATH_INFO: `/v3/projects/faf0c6e98206493ea25654f44c9ae563` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.607191 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf0c6e98206493ea25654f44c9ae563)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.609461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.620494 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1418/2871] 10.222.0.70 () {60 vars in 1153 bytes} [Tue Aug 9 19:24:32 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:32.623423 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd0868-20ab-4c7c-b550-f3410ef598f4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.665663 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf0c6e98206493ea25654f44c9ae563', 'request_id': 'req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.673224 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf0c6e98206493ea25654f44c9ae563', 'request_id': 'req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.681833 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf0c6e98206493ea25654f44c9ae563', 'request_id': 'req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.696158 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] The token cache is being invalidate because project faf0c6e98206493ea25654f44c9ae563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:32.696672 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.697374 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.700630 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1454/2872] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:32 2022] DELETE /identity/v3/projects/faf0c6e98206493ea25654f44c9ae563 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:32.700814 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd0868-20ab-4c7c-b550-f3410ef598f4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.703903 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.705406 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.764617 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:32.771354 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.772201 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.772883 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.773367 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.774144 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.776535 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.781748 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca8eb-09b2-4916-88fd-ebf3169a1c07 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.787844 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.788590 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.789072 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.789438 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] PATH_INFO: `/v3/users/66ab32c245f04b5baab7426dfe1585d6` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.790724 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ab32c245f04b5baab7426dfe1585d6)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.792768 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.806328 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1419/2873] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Aug 9 19:24:32 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 19:24:32.807700 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.809913 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.822661 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d69630-e778-459b-a75b-231d942adc89 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.828995 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bbbd2-29da-41af-8310-d5e294c9099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.829712 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bbbd2-29da-41af-8310-d5e294c9099e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.829987 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bbbd2-29da-41af-8310-d5e294c9099e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.830460 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bbbd2-29da-41af-8310-d5e294c9099e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.831175 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bbbd2-29da-41af-8310-d5e294c9099e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.833019 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bbbd2-29da-41af-8310-d5e294c9099e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.848732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1420/2874] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 19:24:32.850309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bbbd2-29da-41af-8310-d5e294c9099e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.850968 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ab32c245f04b5baab7426dfe1585d6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.860509 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ab32c245f04b5baab7426dfe1585d6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.863952 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bbbd2-29da-41af-8310-d5e294c9099e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.869735 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ab32c245f04b5baab7426dfe1585d6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.870380 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ab32c245f04b5baab7426dfe1585d6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.871375 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.871678 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c0025629-de21-44c9-b941-288d669c8b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ab32c245f04b5baab7426dfe1585d6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:32.872262 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.872642 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.873258 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.874519 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:32.876529 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:32.888796 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1421/2875] 10.222.0.70 () {58 vars in 1122 bytes} [Tue Aug 9 19:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 19:24:32.890516 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1455/2876] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:32 2022] DELETE /identity/v3/users/66ab32c245f04b5baab7426dfe1585d6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 19:24:32.892717 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:32.894530 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:32.907956 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f68af3-13ad-4563-ad76-85026c687613 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:32.977617 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:32.978293 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:32.978576 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:32.978854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] PATH_INFO: `/v3/projects/3bc4d56a7a464c06970f459a412576a3/users/018c3f6da2cf4cb8ad323a476e0bd824/roles/46921b1a8b1a4d2d9ad0dd7070581618` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:32.992640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:32.997935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc4d56a7a464c06970f459a412576a3, user_id=018c3f6da2cf4cb8ad323a476e0bd824, role_id=46921b1a8b1a4d2d9ad0dd7070581618)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:33.001747 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:33.021049 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1422/2877] 10.222.0.70 () {60 vars in 1483 bytes} [Tue Aug 9 19:24:32 2022] PUT /identity/v3/projects/3bc4d56a7a464c06970f459a412576a3/users/018c3f6da2cf4cb8ad323a476e0bd824/roles/46921b1a8b1a4d2d9ad0dd7070581618 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:33.040243 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc933-834d-49e0-bf59-be511a5c6b64 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:33.041854 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc933-834d-49e0-bf59-be511a5c6b64 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:33.043571 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc933-834d-49e0-bf59-be511a5c6b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:33.079187 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ae7dc933-834d-49e0-bf59-be511a5c6b64 None None] MFA Rules not processed for user `018c3f6da2cf4cb8ad323a476e0bd824`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:33.080837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ae7dc933-834d-49e0-bf59-be511a5c6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:33.115885 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:33.118619 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:33.185151 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1456/2878] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:33.197997 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dc933-834d-49e0-bf59-be511a5c6b64 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:33.201365 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ae7dc933-834d-49e0-bf59-be511a5c6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:33.246734 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2674b6-c466-4d18-9336-503907a64235 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:33.251659 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:33.252295 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:33.252732 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:33.253004 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] PATH_INFO: `/v3/projects/c0a5d36de68946cd8d14aac3fdbdbd10` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:33.254584 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a5d36de68946cd8d14aac3fdbdbd10)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:33.256028 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:33.293804 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dc933-834d-49e0-bf59-be511a5c6b64 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:33.297949 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42bdb0-b237-43d2-9307-8be65dce3332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00be9b6d551a4ba39718a44101ecf9ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:33.298679 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42bdb0-b237-43d2-9307-8be65dce3332 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:33.299080 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42bdb0-b237-43d2-9307-8be65dce3332 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:33.299461 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42bdb0-b237-43d2-9307-8be65dce3332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:33.309355 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42bdb0-b237-43d2-9307-8be65dce3332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:33.311392 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42bdb0-b237-43d2-9307-8be65dce3332 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:33.317467 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:33.317802 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a5d36de68946cd8d14aac3fdbdbd10', 'request_id': 'req-98076cd4-459d-4e40-9f7b-08fda0f0a5da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:33.323284 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1457/2879] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 19:24:33.324633 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a5d36de68946cd8d14aac3fdbdbd10', 'request_id': 'req-98076cd4-459d-4e40-9f7b-08fda0f0a5da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:33.334493 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a5d36de68946cd8d14aac3fdbdbd10', 'request_id': 'req-98076cd4-459d-4e40-9f7b-08fda0f0a5da'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:33.347415 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] The token cache is being invalidate because project c0a5d36de68946cd8d14aac3fdbdbd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:33.347819 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:33.348615 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-98076cd4-459d-4e40-9f7b-08fda0f0a5da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:33.351261 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1423/2880] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:33 2022] DELETE /identity/v3/projects/c0a5d36de68946cd8d14aac3fdbdbd10 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:35.914546 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42bdb0-b237-43d2-9307-8be65dce3332 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:35.926422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42bdb0-b237-43d2-9307-8be65dce3332 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:35.982520 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:35.983143 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:35.983422 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:35.983696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] PATH_INFO: `/v3/users/018c3f6da2cf4cb8ad323a476e0bd824` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:35.984846 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=018c3f6da2cf4cb8ad323a476e0bd824)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:35.986793 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:36.033659 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018c3f6da2cf4cb8ad323a476e0bd824'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.040533 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018c3f6da2cf4cb8ad323a476e0bd824'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.047939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018c3f6da2cf4cb8ad323a476e0bd824'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.048612 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018c3f6da2cf4cb8ad323a476e0bd824'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.049238 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018c3f6da2cf4cb8ad323a476e0bd824'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.066180 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1458/2881] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:24:35 2022] DELETE /identity/v3/users/018c3f6da2cf4cb8ad323a476e0bd824 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:36.070752 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343436c-6d1f-40e2-bfd0-13008eb4ab49 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:36.071224 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343436c-6d1f-40e2-bfd0-13008eb4ab49 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:36.071338 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343436c-6d1f-40e2-bfd0-13008eb4ab49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:36.090729 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6343436c-6d1f-40e2-bfd0-13008eb4ab49 None None] MFA Rules not processed for user `910b0103ce4a49babfe0b9390ae4fdd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 19:24:36.091911 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6343436c-6d1f-40e2-bfd0-13008eb4ab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:36.148541 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1424/2882] 10.222.0.70 () {58 vars in 958 bytes} [Tue Aug 9 19:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 19:24:36.158696 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:36.160119 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:36.217094 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ef4a9-2633-45ec-ae80-a28c625f6aae admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:36.220717 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:36.221312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:36.221588 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:36.221855 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:36.231165 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:36.232626 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:36.241993 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1459/2883] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:24:36.509623 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343436c-6d1f-40e2-bfd0-13008eb4ab49 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:24:36.511157 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6343436c-6d1f-40e2-bfd0-13008eb4ab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:24:36.526421 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343436c-6d1f-40e2-bfd0-13008eb4ab49 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:24:36.531924 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:24:36.532596 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:24:36.532935 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:24:36.534187 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] PATH_INFO: `/v3/projects/3bc4d56a7a464c06970f459a412576a3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:24:36.534512 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc4d56a7a464c06970f459a412576a3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:24:36.536177 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:24:36.557961 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:36.589670 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc4d56a7a464c06970f459a412576a3', 'request_id': 'req-6b452dfd-77cc-4df6-a914-5c5f294229aa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.595104 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc4d56a7a464c06970f459a412576a3', 'request_id': 'req-6b452dfd-77cc-4df6-a914-5c5f294229aa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.602246 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc4d56a7a464c06970f459a412576a3', 'request_id': 'req-6b452dfd-77cc-4df6-a914-5c5f294229aa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.614022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] The token cache is being invalidate because project 3bc4d56a7a464c06970f459a412576a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:24:36.614309 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.614920 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:24:36.616373 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1425/2884] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:24:36 2022] DELETE /identity/v3/projects/3bc4d56a7a464c06970f459a412576a3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:24:46.237715 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:24:46.611113 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:26:05.906430 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:05.907889 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:06.004758 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904a01b-da6b-4d72-8732-900a87643218 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:06.097407 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:06.098021 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:06.098300 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:06.098573 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:06.100348 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:06.114313 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:06.115961 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:06.131310 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1460/2885] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:26:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:26:08.331343 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:08.332653 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:08.354690 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452dfd-77cc-4df6-a914-5c5f294229aa admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:08.361347 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:08.361957 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:08.362239 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:08.362513 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:08.364285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:08.377612 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:08.379196 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:08.396256 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1426/2886] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:26:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:26:09.562964 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:09.573837 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19023a5d-1da2-4e8c-9c9d-2eae0e5a4572 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:09.580298 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48410d57fe0d489cbb07cfca58ca7b30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:09.580918 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:09.581201 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:09.581479 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:09.583271 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:09.643981 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:09.645521 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:09.654529 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1461/2887] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:26:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:26:09.906524 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:09.907919 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:09.964723 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af43c6-3555-4c40-85c2-f1472eed2ad8 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:09.969225 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31183ac0fdd642e6a1a7578899aa9fdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:09.969954 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:09.970232 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:09.970508 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:09.972387 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:10.036856 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:10.038322 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:10.048948 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1427/2888] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:26:16.390184 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:16.392420 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:16.420122 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:26:16.450963 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:16.454191 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:16.472895 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f04da-d08b-49a4-a3d9-8b4a5910ff50 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:16.477640 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:16.478276 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743e53-6954-43ff-b80a-c3fb89a7f4ce service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:16.478417 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:16.478765 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:16.479105 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] PATH_INFO: `/v3/users/b66be721908a4eb1a136d416a4d8b2f9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:16.480348 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=b66be721908a4eb1a136d416a4d8b2f9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:16.482461 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:16.483390 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:16.483990 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:16.484261 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:16.484560 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:16.486340 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:16.498150 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:16.500652 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:16.511840 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1462/2889] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:26:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:26:16.541132 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b66be721908a4eb1a136d416a4d8b2f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.550022 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b66be721908a4eb1a136d416a4d8b2f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.557342 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b66be721908a4eb1a136d416a4d8b2f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.557971 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b66be721908a4eb1a136d416a4d8b2f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.558577 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b66be721908a4eb1a136d416a4d8b2f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.575368 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1428/2890] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:26:16 2022] DELETE /identity/v3/users/b66be721908a4eb1a136d416a4d8b2f9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:26:16.757909 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:16.772227 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f05eb1-4bad-48cf-92cd-351c455a6fb9 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:16.834312 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:16.834939 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:16.835219 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:16.835491 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] PATH_INFO: `/v3/projects/57933942d654420985e7bc588dc98c1f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:16.836706 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57933942d654420985e7bc588dc98c1f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:16.838604 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:16.884623 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57933942d654420985e7bc588dc98c1f', 'request_id': 'req-14786cbc-ff74-4ada-b52f-1895aa805dc3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.890687 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57933942d654420985e7bc588dc98c1f', 'request_id': 'req-14786cbc-ff74-4ada-b52f-1895aa805dc3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.897936 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57933942d654420985e7bc588dc98c1f', 'request_id': 'req-14786cbc-ff74-4ada-b52f-1895aa805dc3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.909751 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] The token cache is being invalidate because project 57933942d654420985e7bc588dc98c1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:26:16.910032 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.910664 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:16.912113 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1463/2891] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:26:16 2022] DELETE /identity/v3/projects/57933942d654420985e7bc588dc98c1f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:26:16.916176 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:16.917877 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:16.930946 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cce38d-e4c3-429a-ac21-f2ab376f03ba admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:16.935492 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:26:16.988800 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:16.989429 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:16.989704 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:16.989993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] PATH_INFO: `/v3/users/40e5ac7c8b334a46a2bbe2121c8f5566` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:16.991129 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e5ac7c8b334a46a2bbe2121c8f5566)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:16.993065 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:17.038532 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e5ac7c8b334a46a2bbe2121c8f5566'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.046854 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e5ac7c8b334a46a2bbe2121c8f5566'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.053975 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e5ac7c8b334a46a2bbe2121c8f5566'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.054657 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e5ac7c8b334a46a2bbe2121c8f5566'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.055547 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e5ac7c8b334a46a2bbe2121c8f5566'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.078458 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1429/2892] 10.222.0.70 () {58 vars in 1224 bytes} [Tue Aug 9 19:26:16 2022] DELETE /identity/v3/users/40e5ac7c8b334a46a2bbe2121c8f5566 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:26:17.265800 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:17.267935 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:17.282401 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14786cbc-ff74-4ada-b52f-1895aa805dc3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:17.324570 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:26:17.341892 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910b0103ce4a49babfe0b9390ae4fdd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de422168330743c9990b96b3dab6b6f7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:17.343054 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:17.343448 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:17.343721 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] PATH_INFO: `/v3/projects/f90a8128151c4216bf2fc878233ce45a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:17.344864 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90a8128151c4216bf2fc878233ce45a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:17.346791 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:17.392684 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90a8128151c4216bf2fc878233ce45a', 'request_id': 'req-870e5207-0f40-4754-a221-f21e7f456d80'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.398387 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90a8128151c4216bf2fc878233ce45a', 'request_id': 'req-870e5207-0f40-4754-a221-f21e7f456d80'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.406725 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90a8128151c4216bf2fc878233ce45a', 'request_id': 'req-870e5207-0f40-4754-a221-f21e7f456d80'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.417994 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] The token cache is being invalidate because project f90a8128151c4216bf2fc878233ce45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 19:26:17.418285 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.418923 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-870e5207-0f40-4754-a221-f21e7f456d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 19:26:17.420415 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1464/2893] 10.222.0.70 () {58 vars in 1233 bytes} [Tue Aug 9 19:26:17 2022] DELETE /identity/v3/projects/f90a8128151c4216bf2fc878233ce45a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 19:26:27.074573 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:26:27.414760 dv-cinder-852654-1-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 19:26:59.558694 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 19:26:59.560114 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:59.632422 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a2ed2-ee8b-4eae-805f-8fef24413867 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 19:26:59.690680 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0cf92-1455-43c7-bfad-6fa78330956c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79078cd57816487e8b2b85856c13d5a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32d2105c511b40ed9df707fde095a0b6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 19:26:59.691285 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0cf92-1455-43c7-bfad-6fa78330956c service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 19:26:59.691565 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0cf92-1455-43c7-bfad-6fa78330956c service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 19:26:59.691837 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0cf92-1455-43c7-bfad-6fa78330956c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 19:26:59.693588 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-60f0cf92-1455-43c7-bfad-6fa78330956c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 19:26:59.705370 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0cf92-1455-43c7-bfad-6fa78330956c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 19:26:59.706993 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0cf92-1455-43c7-bfad-6fa78330956c service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 19:26:59.717873 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1430/2894] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Aug 9 19:26:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 19:27:09.712864 dv-cinder-852654-1-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=58 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}